-- Logs begin at Tue 2022-07-26 13:51:31 UTC, end at Tue 2022-07-26 15:27:11 UTC. -- Jul 26 14:38:53.035921 dv-cinder-850668-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 26 14:38:53.052757 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 26 14:38:53.056044 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 26 14:38:53 2022] *** Jul 26 14:38:53.056044 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 26 14:38:53.056044 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 26 14:38:53.056044 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: nodename: dv-cinder-850668-2-iscsi Jul 26 14:38:53.056044 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: machine: x86_64 Jul 26 14:38:53.056044 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: clock source: unix Jul 26 14:38:53.056044 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: pcre jit disabled Jul 26 14:38:53.056044 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: detected number of CPU cores: 4 Jul 26 14:38:53.056044 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: current working directory: / Jul 26 14:38:53.056044 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: detected binary path: /usr/bin/uwsgi-core Jul 26 14:38:53.056044 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: your processes number limit is 47680 Jul 26 14:38:53.056044 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: your memory page size is 4096 bytes Jul 26 14:38:53.064975 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: detected max file descriptor number: 2048 Jul 26 14:38:53.064975 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: lock engine: pthread robust mutexes Jul 26 14:38:53.064975 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: thunder lock: enabled Jul 26 14:38:53.064975 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 26 14:38:53.064975 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 26 14:38:53.126840 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: Python main interpreter initialized at 0x565505adc9c0 Jul 26 14:38:53.126840 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: python threads support enabled Jul 26 14:38:53.126840 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: your server socket listen backlog is limited to 100 connections Jul 26 14:38:53.126840 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: your mercy for graceful operations on workers is 90 seconds Jul 26 14:38:53.127427 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: mapped 403077 bytes (393 KB) for 2 cores Jul 26 14:38:53.127629 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: *** Operational MODE: preforking *** Jul 26 14:38:53.128155 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: *** uWSGI is running in multiple interpreter mode *** Jul 26 14:38:53.128155 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: spawned uWSGI master process (pid: 92195) Jul 26 14:38:53.128460 dv-cinder-850668-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 26 14:38:53.130972 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: spawned uWSGI worker 1 (pid: 92208, cores: 1) Jul 26 14:38:53.130972 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: spawned uWSGI worker 2 (pid: 92209, cores: 1) Jul 26 14:38:53.130972 dv-cinder-850668-2-iscsi devstack@keystone.service[92195]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 26 14:39:00.224023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 14:39:00.224023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: from cryptography.utils import int_from_bytes Jul 26 14:39:00.225271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 14:39:00.225271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: from cryptography.utils import int_from_bytes Jul 26 14:39:00.280693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 14:39:00.280916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 14:39:00.281051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] command line args: [] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 14:39:00.281320 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 14:39:00.281536 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ================================================================================ {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 14:39:00.281778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] admin_token = **** {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.282021 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] config_dir = [] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.282206 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.282384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] config_source = [] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.282607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] debug = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.282808 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.284712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.284712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.284712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.284712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.284712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] list_limit = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.284712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] log_config_append = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.284712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.284712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] log_dir = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.285276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] log_file = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.285276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.285551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.285766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.285997 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.286178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.286356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.286599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.286842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.287074 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.287304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.287568 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.287799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.288045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.288287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.288605 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.290114 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.94/identity {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.290463 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] publish_errors = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.290867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.291109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.291310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.291587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.291813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.292048 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] standard_threads = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.292280 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.292523 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.292699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.292873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] use_journal = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.293047 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] use_json = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.293260 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] use_stderr = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.293442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] use_syslog = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.293685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.293944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.294181 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.294550 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.294771 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.295019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.295362 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.295727 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.295970 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.296217 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.296466 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.296716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.296948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.297203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.297437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.297684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.297927 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.298161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.298415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.298667 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.298909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.299142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.299394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.299689 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.299946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.300193 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.300420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.300650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.300880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.301111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.301359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.301601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.301836 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.302055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.302296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.302537 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.302770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.303015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.303243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.303517 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.303734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.303962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.304190 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.304589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.304908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.305171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.305641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.305968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.306152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.306383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.306618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.306854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.307101 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.307331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.307657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.307845 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.308059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.308292 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.308525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.308753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.309002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.309236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.309473 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.309767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.309953 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.310187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.310417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.310651 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.310885 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.311244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.311379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.311655 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.311906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.312137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.312371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.312605 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.312849 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.313083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.313315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.313559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.313794 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.314094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.314270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.314448 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.314677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.314941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.315178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.315416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.315679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.316025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.316166 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.316373 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.316603 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.316832 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.317059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.317289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.317521 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.317747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.317986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.318218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.318464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.318697 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.318949 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.319207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.319446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.319701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] auth.external = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.319934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.320747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.321103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.321545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] auth.password = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.321813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] auth.token = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.322058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.322350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.322543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.322773 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.323031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.323263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.323548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.323800 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] credential.caching = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.324037 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.324279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.324526 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.324871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.325017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.325302 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.325482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.325730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.325965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.326268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.326490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.326733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.326993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.327234 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.327530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.327749 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] federation.caching = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.327986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.328230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.328474 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.328716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.328974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.329210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.329464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.329698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.329956 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.330189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.330451 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.330686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity.caching = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.330934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.331179 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.331412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.331678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.331927 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.332157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.332397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.332644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.332898 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.333130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.333361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.333589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.333832 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.334077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.334320 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.334614 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.334814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.335082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.335318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.335578 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.336393 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.336850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.337290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.337735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.338187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.338625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.339084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.339534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.339994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.340464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.340902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.341348 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.341808 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.342164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.342630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.343045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.343529 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.343953 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.344408 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.344855 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.345296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.345759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.346223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.346545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.346977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.347449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.347950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.348483 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.348829 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.349269 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.349703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.350160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.350633 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.351056 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.352143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.352564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.353041 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.353486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.353921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.354366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.354796 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.355243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.355710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.356181 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.356646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.357107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.357802 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.358268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.358711 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.359171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.359642 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.360089 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.360544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.360993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.361530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.361900 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.362338 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.362715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.363147 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.363640 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.364079 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.364514 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.364955 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.365631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.365977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.366299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.366740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.367395 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] resource.caching = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.368408 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.368876 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.369421 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.369789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.370257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.370703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.371161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.371640 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.372092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.372527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] role.caching = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.372989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] role.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.373388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.373866 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.374329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.374784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.375248 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.375738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.376197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.376654 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.377121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.377566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.378019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.378472 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.378802 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.379254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.380064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.380504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.380975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.381430 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.381882 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.382198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.382490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.383487 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.384047 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.384436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.384875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.385317 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.385747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.386205 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.386668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.387123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.387669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.388034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.388413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.388707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] token.caching = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.389027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.389491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.389929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.390411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.390871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.391309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.391754 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.392015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.392483 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.392928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.393378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.393809 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.394269 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.395277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.395277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.395677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.396152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.396645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.connection = **** {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.397103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.397566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.398015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.399029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.399531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.400106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.400465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.400875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.401347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.401791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.403285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.403285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.403285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.403740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.404030 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.404443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.404916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.405334 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.405587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92209) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 14:39:00.464486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.464769 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.464970 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.481921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.537777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.538083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.616777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.660051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 14:39:00.660051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: from cryptography.utils import int_from_bytes Jul 26 14:39:00.660861 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 14:39:00.660861 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: from cryptography.utils import int_from_bytes Jul 26 14:39:00.674348 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.674593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.723255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 14:39:00.723534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 14:39:00.723696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] command line args: [] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 14:39:00.723965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 14:39:00.724143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ================================================================================ {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 14:39:00.724402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] admin_token = **** {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.724674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] config_dir = [] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.724839 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.725021 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] config_source = [] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.725203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] debug = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.725450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.725653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.725831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.726081 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.726994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.727203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] list_limit = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.727577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] log_config_append = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.727747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.727934 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] log_dir = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.728136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] log_file = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.728345 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.728594 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.728840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.729103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.729274 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.729455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.729697 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.729941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.730174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.730449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.730644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.730911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.731128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.731375 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.731695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.732287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.94/identity {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.732431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] publish_errors = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.732657 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.732846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.733027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.733285 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.733531 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.733849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] standard_threads = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.734110 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.734342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.734532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.734728 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] use_journal = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.734843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] use_json = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.735009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] use_stderr = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.735171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] use_syslog = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.735431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 14:39:00.735767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.736107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.736377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.736620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.736872 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.737115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.737621 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.737621 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.737843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.738084 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.738335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.738607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.738796 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.739053 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.739327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.739584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.740414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.740414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.740414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.741156 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.741156 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.741746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.741746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.742065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.742326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.742566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.742799 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.743043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.743283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.743573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.743821 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.744056 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.744296 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.744539 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.744794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.745028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.745311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.745550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.745843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.746084 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.746300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.746426 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.746522 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.746635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.746753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.746835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.746943 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.747062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.747146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.747291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.747416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.747590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.747746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.747885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.748122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.748364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.748598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.748847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.749063 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.749302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.749564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.749928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.750218 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.750461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.750698 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.750960 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.751189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.751435 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.751694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.751973 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.752284 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.752461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.752706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.752995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.753235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.753481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.753745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.753991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.754232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.754498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.754700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.754955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.755210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.755436 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.755691 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.755932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.756153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.756392 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.756612 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.756847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.757116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.757354 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.757665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.757902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.758159 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.758385 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.758671 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.758990 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.759915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.759915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.759915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.760442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.760442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.760442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.760986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.760986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.761386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.761386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.761712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.761890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.762109 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] auth.external = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.762337 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.762635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.762868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.763101 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] auth.password = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.763328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] auth.token = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.763593 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.763830 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.764079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.764309 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.764557 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.764808 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.765039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.765275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] credential.caching = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.765556 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.765792 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.766022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.766260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.766525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.766768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.767015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.767340 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.767665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.767924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.768253 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.768507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.768768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.769045 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.769272 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.769509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] federation.caching = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.769746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.769994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.770246 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.770476 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.770724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.770961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.771218 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.771453 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.771893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.772127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.772346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.772561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.772775 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.773001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.773975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.774393 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.774683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.775115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.775409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity.caching = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.775753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.775753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.776084 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.776256 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.776472 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.776821 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.777073 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.777337 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.777602 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.777851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.778087 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.778325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.778573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.778832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.779086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.779360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.779610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.779888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.780230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.780467 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.780803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.781029 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.781287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.781529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.781758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.781991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.782251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.782486 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.782733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.782992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.783229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.783511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.783749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.783986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.784227 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.784488 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.784703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.784951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.785223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.785425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.785673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.785931 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.786169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.786404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.786529 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.786764 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.786996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.787176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.787322 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.787663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.787847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.788028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.788261 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.788533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.789940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.789940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.790239 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.790394 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.790635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.790872 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.791107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.791347 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.791612 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.791847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.792099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.792348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.792590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.792831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.793080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.793314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.793631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.793990 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.794233 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.794538 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.794882 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.795136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.795562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.795762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.796039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.796264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.796510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.796791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.797151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.797544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.797951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.798324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.798703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.799038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.799229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] resource.caching = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.799602 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.800039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.800461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.800917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.801382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.801701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.801961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.802255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.802549 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.802821 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] role.caching = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.803119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] role.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.803405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.803671 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.803803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.803968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.804155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.804315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.804315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.804851 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.805040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.805207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.805404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.805599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.805599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92209) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.806036 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.806222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.806611 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.807122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.807635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.808149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.808614 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.809098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.809923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.810443 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.810909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.811361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.811894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.812358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.812804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.813278 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.813817 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.814330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.814828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.815376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.815848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.816330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.816813 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.817299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.817779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.818243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.818695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.819214 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] token.caching = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.819623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.820183 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.827701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.827701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.827701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.827701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.827701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.827701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.827701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.827701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.827701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.828998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.829203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.829386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.829588 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 14:39:00.829721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.829852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.829998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.830126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 14:39:00.830126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.830361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.830496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.830625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.830767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.830767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.830996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.connection = **** {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.830996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.830996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.830996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.830996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.830996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.830996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.830996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.830996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.831912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.831912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.831912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.831912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.832359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.832473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.832894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.833369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.833815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.834243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 14:39:00.834560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92208) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 14:39:00.840152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.840464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.841442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.841442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.848622 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:00.855679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:00.855974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.856313 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.856721 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.857125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.857531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.857822 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.858192 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.858473 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.858746 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.859009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.884652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.885035 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.885239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.885414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.886940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:00.887219 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.893714 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:00.894113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.894466 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.895905 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.905712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:00.906134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.911084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:00.911391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.911694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.912006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.912281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.912569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.912835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.913113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.920942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.921392 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:00.934021 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.934293 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.938608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:00.943180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.943516 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.949680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:00.953733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:00.957649 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.958094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.958506 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.958943 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.959235 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.959567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.981186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:00.981671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.982132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.982659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.995509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.998921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:00.999393 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.004962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.005463 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.005998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.006492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.006994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.030168 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.030612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.030906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.031182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.031582 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.032023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.032495 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.050811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.054126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.058389 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.061406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.063099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.063384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.063715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.071195 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.073533 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.076754 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.077063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.077346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.077615 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.086222 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.086721 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.087107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.094376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92209) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.094669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.094947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.095230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.095533 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.095817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.096087 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.096372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.096772 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.097054 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.097341 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.097602 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.097876 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.098139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92209) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.128443 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.136669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x565505adc9c0 pid: 92209 (default app) Jul 26 14:39:01.140318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746df12-aa79-4a1e-9e7d-cafd707ccfc8 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:01.140605 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746df12-aa79-4a1e-9e7d-cafd707ccfc8 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:01.140912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746df12-aa79-4a1e-9e7d-cafd707ccfc8 None None] PATH_INFO: `/v3` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:01.145620 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1/1] 10.222.0.94 () {50 vars in 791 bytes} [Tue Jul 26 14:39:01 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:01.203661 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.204071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.299344 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.302292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.302944 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.303299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.303744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.304175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.330628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.330960 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.331267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.331636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.331935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.332374 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.332699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.341900 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.345484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.345714 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.345903 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.346114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.346322 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.346530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.346738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92208) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 14:39:01.358125 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.358344 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 14:39:01.358664 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.358869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 14:39:01.359142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.359393 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.359709 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.359972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.370753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.371037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.371389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.371703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.379449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.382838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.383310 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.383746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.384116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.384493 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.384822 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.385159 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.385537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.385870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.386190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.386511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.414963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.416811 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.417109 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.422433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.422844 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.423211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.431215 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.431549 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.436257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.436600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.436878 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.437284 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.437604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.437887 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.438204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.438490 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.456632 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.457008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.461714 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.465203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.467372 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.472057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.475524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.480347 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.480765 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.481285 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.481678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.482080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.482455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.501304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.501662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.502010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.502359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.511859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.513805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.514213 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.517445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.517853 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.518211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.518603 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.518951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.532277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.532645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.532926 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.533203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.533582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.533858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.534143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.551956 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.555812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.559279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.561363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.562960 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.563239 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.563543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.571635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.573864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.577352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.577352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.579054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.579054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.589290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.589700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.590064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.597915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92208) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 14:39:01.598377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.598662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.599030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.599322 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.599623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.599898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.600184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.600453 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.600721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.600996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.601317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.602288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.602679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92208) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 14:39:01.646983 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x565505adc9c0 pid: 92208 (default app) Jul 26 14:39:11.376263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31944834-f042-4241-b257-4c0e5a317e10 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:11.376856 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31944834-f042-4241-b257-4c0e5a317e10 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:11.376984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31944834-f042-4241-b257-4c0e5a317e10 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:11.378724 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 2/2] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:39:11.392939 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b4a62-e552-4717-954a-611459e1200d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:11.393542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b4a62-e552-4717-954a-611459e1200d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:11.393876 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b4a62-e552-4717-954a-611459e1200d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:11.541962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: INFO dbcounter [None req-826b4a62-e552-4717-954a-611459e1200d None None] Registered counter for database keystone Jul 26 14:39:11.579527 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG oslo_db.sqlalchemy.engines [None req-826b4a62-e552-4717-954a-611459e1200d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92208) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 14:39:11.587190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writer thread running {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 14:39:11.791662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG passlib.handlers.bcrypt [None req-826b4a62-e552-4717-954a-611459e1200d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92208) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 14:39:11.792316 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG passlib.handlers.bcrypt [None req-826b4a62-e552-4717-954a-611459e1200d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92208) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 14:39:11.854932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-826b4a62-e552-4717-954a-611459e1200d None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:11.856989 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-826b4a62-e552-4717-954a-611459e1200d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:12.019880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1/3] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 635 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 26 14:39:12.190688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31944834-f042-4241-b257-4c0e5a317e10 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:12.268477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: INFO dbcounter [None req-31944834-f042-4241-b257-4c0e5a317e10 None None] Registered counter for database keystone Jul 26 14:39:12.293046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG oslo_db.sqlalchemy.engines [None req-31944834-f042-4241-b257-4c0e5a317e10 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92209) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 14:39:12.301443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writer thread running {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 14:39:12.439891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31944834-f042-4241-b257-4c0e5a317e10 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:12.445033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0fae0-0694-4e56-8127-8107cce66f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:12.445664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0fae0-0694-4e56-8127-8107cce66f02 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:12.446006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0fae0-0694-4e56-8127-8107cce66f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:12.446361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0fae0-0694-4e56-8127-8107cce66f02 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:12.456211 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0fae0-0694-4e56-8127-8107cce66f02 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:12.511721 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e5a0fae0-0694-4e56-8127-8107cce66f02 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 14:39:12.512191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0fae0-0694-4e56-8127-8107cce66f02 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:12.524248 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 3/4] 10.222.0.94 () {58 vars in 1324 bytes} [Tue Jul 26 14:39:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 14:39:12.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b4a62-e552-4717-954a-611459e1200d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:12.548446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b4a62-e552-4717-954a-611459e1200d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:12.553790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4f335-505e-4adc-9cef-ab85101705cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:12.555738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4f335-505e-4adc-9cef-ab85101705cd admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:12.556108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4f335-505e-4adc-9cef-ab85101705cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:12.556469 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4f335-505e-4adc-9cef-ab85101705cd admin admin] PATH_INFO: `/v3/projects/382829af20d649b9beb375e3c3cc2d50` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:12.558273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b4f335-505e-4adc-9cef-ab85101705cd admin admin] RBAC: Authorizing `identity:get_project(project_id=382829af20d649b9beb375e3c3cc2d50)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:12.613577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e9b4f335-505e-4adc-9cef-ab85101705cd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 14:39:12.613914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b4f335-505e-4adc-9cef-ab85101705cd admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:12.618968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 2/5] 10.222.0.94 () {56 vars in 1207 bytes} [Tue Jul 26 14:39:12 2022] GET /identity/v3/projects/382829af20d649b9beb375e3c3cc2d50 => generated 344 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:15.046006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383295c-be25-4207-9492-02e39f096015 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:15.046817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383295c-be25-4207-9492-02e39f096015 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:15.046963 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383295c-be25-4207-9492-02e39f096015 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:15.051240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 4/6] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:39:15.058939 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612af4e7-cf2e-4e97-8d96-a77b4d4f917e None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:15.059521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612af4e7-cf2e-4e97-8d96-a77b4d4f917e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:15.059638 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612af4e7-cf2e-4e97-8d96-a77b4d4f917e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:15.083178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-612af4e7-cf2e-4e97-8d96-a77b4d4f917e None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:15.084851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-612af4e7-cf2e-4e97-8d96-a77b4d4f917e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:15.094390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 3/7] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 26 14:39:15.134942 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383295c-be25-4207-9492-02e39f096015 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:15.157811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383295c-be25-4207-9492-02e39f096015 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:15.165737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce1d43-7e8f-45c8-82dc-49c3ef231a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:15.166767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce1d43-7e8f-45c8-82dc-49c3ef231a85 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:15.167264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce1d43-7e8f-45c8-82dc-49c3ef231a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:15.167789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce1d43-7e8f-45c8-82dc-49c3ef231a85 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:15.184436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce1d43-7e8f-45c8-82dc-49c3ef231a85 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:15.188013 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce1d43-7e8f-45c8-82dc-49c3ef231a85 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:15.206092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 5/8] 10.222.0.94 () {58 vars in 1324 bytes} [Tue Jul 26 14:39:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 14:39:15.215667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612af4e7-cf2e-4e97-8d96-a77b4d4f917e None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:15.227001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612af4e7-cf2e-4e97-8d96-a77b4d4f917e None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:15.232899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3878ff-e976-411e-b3ef-dd7875dc44a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:15.233570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3878ff-e976-411e-b3ef-dd7875dc44a6 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:15.233849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3878ff-e976-411e-b3ef-dd7875dc44a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:15.234119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3878ff-e976-411e-b3ef-dd7875dc44a6 admin admin] PATH_INFO: `/v3/users/31dac45eb8764daf9a148a750fdc78a0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:15.235654 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3878ff-e976-411e-b3ef-dd7875dc44a6 admin admin] RBAC: Authorizing `identity:get_user(user_id=31dac45eb8764daf9a148a750fdc78a0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:15.237576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3878ff-e976-411e-b3ef-dd7875dc44a6 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:15.239272 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 4/9] 10.222.0.94 () {56 vars in 1198 bytes} [Tue Jul 26 14:39:15 2022] GET /identity/v3/users/31dac45eb8764daf9a148a750fdc78a0 => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:17.534853 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5196f41-0bdf-4541-b094-3d0c1e958281 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:17.535398 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5196f41-0bdf-4541-b094-3d0c1e958281 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:17.535880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5196f41-0bdf-4541-b094-3d0c1e958281 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:17.538792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 6/10] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:17.547923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ce7a8-b193-4724-8daa-67b7948246cf None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:17.548501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ce7a8-b193-4724-8daa-67b7948246cf None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:17.548852 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ce7a8-b193-4724-8daa-67b7948246cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:17.576517 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbeb2a-1cd1-4b45-b34c-472d9f2be112 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:17.576947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbeb2a-1cd1-4b45-b34c-472d9f2be112 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:17.577338 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbeb2a-1cd1-4b45-b34c-472d9f2be112 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:17.579075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 7/11] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:39:17.581636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-923ce7a8-b193-4724-8daa-67b7948246cf None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:17.583813 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-923ce7a8-b193-4724-8daa-67b7948246cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:17.586063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fbffb-2ef2-4ab4-a249-a550de48ea58 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:17.586352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fbffb-2ef2-4ab4-a249-a550de48ea58 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:17.586656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fbffb-2ef2-4ab4-a249-a550de48ea58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:17.635267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG passlib.handlers.bcrypt [None req-233fbffb-2ef2-4ab4-a249-a550de48ea58 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92209) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 14:39:17.635764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG passlib.handlers.bcrypt [None req-233fbffb-2ef2-4ab4-a249-a550de48ea58 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92209) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 14:39:17.684018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-233fbffb-2ef2-4ab4-a249-a550de48ea58 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:17.685705 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-233fbffb-2ef2-4ab4-a249-a550de48ea58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:17.721268 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 5/12] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:17.747766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ce7a8-b193-4724-8daa-67b7948246cf None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:17.800782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:17.819169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 8/13] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:39:17.844514 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fbffb-2ef2-4ab4-a249-a550de48ea58 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:17.894433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ce7a8-b193-4724-8daa-67b7948246cf None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:17.955337 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430e5dd-8acb-4f8d-aa82-06583699e90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:17.956472 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430e5dd-8acb-4f8d-aa82-06583699e90a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:17.956980 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430e5dd-8acb-4f8d-aa82-06583699e90a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:17.957460 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430e5dd-8acb-4f8d-aa82-06583699e90a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:17.981978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430e5dd-8acb-4f8d-aa82-06583699e90a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-b430e5dd-8acb-4f8d-aa82-06583699e90a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:17.997501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:17.998629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:17.999507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:17.999507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:17.999507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:17.999507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:17.999507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:17.999507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:17.999507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:17.999507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:17.999507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:17.999507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:17.999507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:39:17.999507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:39:17.999507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 6/14] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:39:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:39:18.005209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430e5dd-8acb-4f8d-aa82-06583699e90a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:18.016885 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fbffb-2ef2-4ab4-a249-a550de48ea58 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:18.021654 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:18.074558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e05cd-657c-4671-be53-5db970bb0554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:18.075376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e05cd-657c-4671-be53-5db970bb0554 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:18.076333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e05cd-657c-4671-be53-5db970bb0554 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:18.076333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e05cd-657c-4671-be53-5db970bb0554 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:18.152697 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:18.165309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e05cd-657c-4671-be53-5db970bb0554 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:18.168265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e05cd-657c-4671-be53-5db970bb0554 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:18.174888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430e5dd-8acb-4f8d-aa82-06583699e90a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:18.224784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8499617-ca0b-4910-8ebe-cfdc725417de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:18.226338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8499617-ca0b-4910-8ebe-cfdc725417de None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:18.226915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8499617-ca0b-4910-8ebe-cfdc725417de None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:18.228186 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8499617-ca0b-4910-8ebe-cfdc725417de None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:18.229032 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8499617-ca0b-4910-8ebe-cfdc725417de None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:18.229578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8499617-ca0b-4910-8ebe-cfdc725417de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:18.231624 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8499617-ca0b-4910-8ebe-cfdc725417de None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:18.246551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 7/15] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:18.252784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8499617-ca0b-4910-8ebe-cfdc725417de None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:18.326946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 9/16] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:39:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:39:18.332826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e05cd-657c-4671-be53-5db970bb0554 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:18.337280 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:18.392600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8499617-ca0b-4910-8ebe-cfdc725417de None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:18.435943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52a7aa-7b52-48b8-8508-1d69d9cba0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:18.436618 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52a7aa-7b52-48b8-8508-1d69d9cba0dc None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:18.436901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52a7aa-7b52-48b8-8508-1d69d9cba0dc None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:18.437216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52a7aa-7b52-48b8-8508-1d69d9cba0dc None admin] PATH_INFO: `/v3/users/31dac45eb8764daf9a148a750fdc78a0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:18.438568 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52a7aa-7b52-48b8-8508-1d69d9cba0dc None admin] RBAC: Authorizing `identity:get_user(user_id=31dac45eb8764daf9a148a750fdc78a0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:18.440141 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52a7aa-7b52-48b8-8508-1d69d9cba0dc None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:18.441936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 8/17] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:18 2022] GET /identity/v3/users/31dac45eb8764daf9a148a750fdc78a0 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:18.451831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52a7aa-7b52-48b8-8508-1d69d9cba0dc None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:18.469883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:18.515925 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e05cd-657c-4671-be53-5db970bb0554 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:18.582311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c38c36-bfca-438d-b20a-8037b6d29388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:18.583112 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c38c36-bfca-438d-b20a-8037b6d29388 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:18.583397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c38c36-bfca-438d-b20a-8037b6d29388 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:18.583705 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c38c36-bfca-438d-b20a-8037b6d29388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:18.593553 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52a7aa-7b52-48b8-8508-1d69d9cba0dc None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:18.616112 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c38c36-bfca-438d-b20a-8037b6d29388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:18.618969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c38c36-bfca-438d-b20a-8037b6d29388 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:18.636092 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a8127-7069-4888-9375-4e3888eb34fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:18.636894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8127-7069-4888-9375-4e3888eb34fc None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:18.637263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8127-7069-4888-9375-4e3888eb34fc None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:18.637622 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8127-7069-4888-9375-4e3888eb34fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:18.639683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a8127-7069-4888-9375-4e3888eb34fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:18.641392 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a8127-7069-4888-9375-4e3888eb34fc None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:18.643138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 9/18] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:39:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:18.651022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a8127-7069-4888-9375-4e3888eb34fc None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-d8c38c36-bfca-438d-b20a-8037b6d29388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:39:18.654176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:18.655548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:18.656372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:39:18.656372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:39:18.656372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:39:18.656372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:39:18.656618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 10/19] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:39:18.662772 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c38c36-bfca-438d-b20a-8037b6d29388 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:18.751177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:18.756146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:18.779201 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a8127-7069-4888-9375-4e3888eb34fc None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:18.820403 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf97d3-27b1-4c61-8302-cfd06dac9462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:18.821076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf97d3-27b1-4c61-8302-cfd06dac9462 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:18.821380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf97d3-27b1-4c61-8302-cfd06dac9462 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:18.821884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf97d3-27b1-4c61-8302-cfd06dac9462 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:18.822576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf97d3-27b1-4c61-8302-cfd06dac9462 None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.domain.id=default, user.id=31dac45eb8764daf9a148a750fdc78a0) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:18.822884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf97d3-27b1-4c61-8302-cfd06dac9462 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:18.824420 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf97d3-27b1-4c61-8302-cfd06dac9462 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:18.852174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 10/20] 10.222.0.94 () {56 vars in 1322 bytes} [Tue Jul 26 14:39:18 2022] GET /identity/v3/role_assignments?user.id=31dac45eb8764daf9a148a750fdc78a0&scope.domain.id=default&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 233 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:18.852393 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c38c36-bfca-438d-b20a-8037b6d29388 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:18.907699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb75b8-3e03-4069-8366-964955bab6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:18.908798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb75b8-3e03-4069-8366-964955bab6de None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:18.909289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb75b8-3e03-4069-8366-964955bab6de None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:18.909787 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb75b8-3e03-4069-8366-964955bab6de None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:18.911209 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb75b8-3e03-4069-8366-964955bab6de None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:18.911800 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb75b8-3e03-4069-8366-964955bab6de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:18.915531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb75b8-3e03-4069-8366-964955bab6de None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:18.952389 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 11/21] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:21.804615 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1b218-3896-493d-ac05-1e4a90cd0ae2 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:21.805408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1b218-3896-493d-ac05-1e4a90cd0ae2 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:21.805408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1b218-3896-493d-ac05-1e4a90cd0ae2 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:21.807171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 11/22] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:21.813035 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8d229-fdfe-49d2-a225-a5b23d8f59d7 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:21.813784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8d229-fdfe-49d2-a225-a5b23d8f59d7 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:21.813784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8d229-fdfe-49d2-a225-a5b23d8f59d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:21.848791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-70a8d229-fdfe-49d2-a225-a5b23d8f59d7 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:21.850099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-70a8d229-fdfe-49d2-a225-a5b23d8f59d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:21.950965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 12/23] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:21.987762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1b218-3896-493d-ac05-1e4a90cd0ae2 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:22.228179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1b218-3896-493d-ac05-1e4a90cd0ae2 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:22.320671 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282b4e7-7b1a-4cb6-a684-aebb7b6dcfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:22.321686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282b4e7-7b1a-4cb6-a684-aebb7b6dcfc7 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:22.322151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282b4e7-7b1a-4cb6-a684-aebb7b6dcfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:22.322866 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282b4e7-7b1a-4cb6-a684-aebb7b6dcfc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:22.354834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282b4e7-7b1a-4cb6-a684-aebb7b6dcfc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-c282b4e7-7b1a-4cb6-a684-aebb7b6dcfc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:22.369995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:22.371718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:22.374065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:22.374065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:22.374065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:22.374065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:22.374065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:22.374065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:22.374065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:22.374065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:22.374065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:22.374065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:22.374065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:39:22.374065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:39:22.374065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 12/24] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:39:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:22.380755 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8d229-fdfe-49d2-a225-a5b23d8f59d7 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:22.417362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cf373-10a0-4698-8f3c-52f693b69e43 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:22.417992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cf373-10a0-4698-8f3c-52f693b69e43 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:22.418437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cf373-10a0-4698-8f3c-52f693b69e43 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:22.421550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 13/25] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:22.433295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1105ec8-7575-4a06-9ac9-11c21c55e268 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:22.433756 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1105ec8-7575-4a06-9ac9-11c21c55e268 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:22.434225 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1105ec8-7575-4a06-9ac9-11c21c55e268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:22.455982 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:22.489441 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-d1105ec8-7575-4a06-9ac9-11c21c55e268 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:22.492712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d1105ec8-7575-4a06-9ac9-11c21c55e268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:22.513969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:22.698371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8d229-fdfe-49d2-a225-a5b23d8f59d7 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:22.711914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 14/26] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:39:22.732902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bce3e7-269b-486c-b449-1f77eb917dd8 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:22.733698 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bce3e7-269b-486c-b449-1f77eb917dd8 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:22.734178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bce3e7-269b-486c-b449-1f77eb917dd8 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:22.736872 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 15/27] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:39:22.746668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7307c9a-19e4-44d6-add3-279096ddee30 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:22.747086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7307c9a-19e4-44d6-add3-279096ddee30 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:22.747420 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7307c9a-19e4-44d6-add3-279096ddee30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:22.796327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-a7307c9a-19e4-44d6-add3-279096ddee30 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:22.797862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a7307c9a-19e4-44d6-add3-279096ddee30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:22.823708 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ae41c-3d5e-4cf0-be61-b529cd642365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:22.824933 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ae41c-3d5e-4cf0-be61-b529cd642365 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:22.825432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ae41c-3d5e-4cf0-be61-b529cd642365 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:22.825921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ae41c-3d5e-4cf0-be61-b529cd642365 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:22.827278 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ae41c-3d5e-4cf0-be61-b529cd642365 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:22.827855 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ae41c-3d5e-4cf0-be61-b529cd642365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:22.830651 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ae41c-3d5e-4cf0-be61-b529cd642365 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:22.870154 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 13/28] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 490 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:22.877736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ae41c-3d5e-4cf0-be61-b529cd642365 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:23.062449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 16/29] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:39:23.064330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7307c9a-19e4-44d6-add3-279096ddee30 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:23.122837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ae41c-3d5e-4cf0-be61-b529cd642365 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:23.304970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:23.356017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:23.399877 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7307c9a-19e4-44d6-add3-279096ddee30 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:23.403975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae19f07-afd8-4a66-a8d9-9f8929057596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:23.405441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae19f07-afd8-4a66-a8d9-9f8929057596 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:23.406218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae19f07-afd8-4a66-a8d9-9f8929057596 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:23.406948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae19f07-afd8-4a66-a8d9-9f8929057596 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:23.408577 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae19f07-afd8-4a66-a8d9-9f8929057596 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:23.411341 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae19f07-afd8-4a66-a8d9-9f8929057596 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:23.458524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1f289-9958-421a-a598-ce24609603af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:23.459396 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f289-9958-421a-a598-ce24609603af None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:23.460415 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f289-9958-421a-a598-ce24609603af None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:23.460839 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f289-9958-421a-a598-ce24609603af None admin] PATH_INFO: `/v3/users/31dac45eb8764daf9a148a750fdc78a0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:23.462474 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1f289-9958-421a-a598-ce24609603af None admin] RBAC: Authorizing `identity:get_user(user_id=31dac45eb8764daf9a148a750fdc78a0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:23.464455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1f289-9958-421a-a598-ce24609603af None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:23.467101 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 17/30] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:23 2022] GET /identity/v3/users/31dac45eb8764daf9a148a750fdc78a0 => generated 247 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:23.467875 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1f289-9958-421a-a598-ce24609603af None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:23.674584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1f289-9958-421a-a598-ce24609603af None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:23.719379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 14/31] 10.222.0.94 () {60 vars in 1131 bytes} [Tue Jul 26 14:39:22 2022] POST /identity/v3/roles => generated 219 bytes in 841 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:39:23.719379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6c230-3591-4b6c-92d3-2f236704b1e0 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:23.719379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6c230-3591-4b6c-92d3-2f236704b1e0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:23.719379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6c230-3591-4b6c-92d3-2f236704b1e0 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:23.719379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 15/32] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:23.720319 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6c230-3591-4b6c-92d3-2f236704b1e0 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:23.738326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:23.745975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2c295-b56f-42ca-b949-2faf7cf88ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:23.746802 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2c295-b56f-42ca-b949-2faf7cf88ac9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:23.747211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2c295-b56f-42ca-b949-2faf7cf88ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:23.747585 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2c295-b56f-42ca-b949-2faf7cf88ac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:23.766693 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2c295-b56f-42ca-b949-2faf7cf88ac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:23.768926 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2c295-b56f-42ca-b949-2faf7cf88ac9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-d7e2c295-b56f-42ca-b949-2faf7cf88ac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:39:23.795395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:23.797137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:23.798780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:39:23.798780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:39:23.798780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:39:23.798780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:39:23.798780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 18/33] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:23.798780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ebb2a-50ac-4d7e-ab1a-0a71211ef5a6 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:23.798780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ebb2a-50ac-4d7e-ab1a-0a71211ef5a6 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:23.799448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ebb2a-50ac-4d7e-ab1a-0a71211ef5a6 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:23.804246 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 19/34] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:23.804246 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f73ae-7427-4281-8921-946f6d94aa05 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:23.804246 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f73ae-7427-4281-8921-946f6d94aa05 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:23.804246 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f73ae-7427-4281-8921-946f6d94aa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:23.839241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-eb8f73ae-7427-4281-8921-946f6d94aa05 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:23.841293 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-eb8f73ae-7427-4281-8921-946f6d94aa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:23.970203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6c230-3591-4b6c-92d3-2f236704b1e0 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:23.984652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 20/35] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:23.988046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f73ae-7427-4281-8921-946f6d94aa05 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:24.041558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f43c05-2939-468c-83e8-47d270feea96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:24.042458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f43c05-2939-468c-83e8-47d270feea96 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:24.042719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f43c05-2939-468c-83e8-47d270feea96 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:24.043859 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f43c05-2939-468c-83e8-47d270feea96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:24.044801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f43c05-2939-468c-83e8-47d270feea96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:24.047055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f43c05-2939-468c-83e8-47d270feea96 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:24.048662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 16/36] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:39:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:24.049867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee20fc0-0d2f-4e55-801c-f58261bd7209 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:24.050165 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee20fc0-0d2f-4e55-801c-f58261bd7209 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:24.050515 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee20fc0-0d2f-4e55-801c-f58261bd7209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:24.072177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-7ee20fc0-0d2f-4e55-801c-f58261bd7209 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:24.073851 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-7ee20fc0-0d2f-4e55-801c-f58261bd7209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:24.123842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:24.164602 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f73ae-7427-4281-8921-946f6d94aa05 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:24.189590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:24.201130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 17/37] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:24.202624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee20fc0-0d2f-4e55-801c-f58261bd7209 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:24.235689 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfd34f-e47a-4e56-9576-88ee0e26c44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:24.236569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfd34f-e47a-4e56-9576-88ee0e26c44b None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:24.237428 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfd34f-e47a-4e56-9576-88ee0e26c44b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:24.237878 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfd34f-e47a-4e56-9576-88ee0e26c44b None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:24.239454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cfd34f-e47a-4e56-9576-88ee0e26c44b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:24.239996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cfd34f-e47a-4e56-9576-88ee0e26c44b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:24.242333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cfd34f-e47a-4e56-9576-88ee0e26c44b None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:24.268304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 21/38] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:24.274954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfd34f-e47a-4e56-9576-88ee0e26c44b None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:24.392472 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee20fc0-0d2f-4e55-801c-f58261bd7209 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:24.413504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfd34f-e47a-4e56-9576-88ee0e26c44b None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:24.437582 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c2acf-adb9-425b-8c20-c665018c84a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:24.438433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c2acf-adb9-425b-8c20-c665018c84a2 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:24.438721 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c2acf-adb9-425b-8c20-c665018c84a2 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:24.439101 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c2acf-adb9-425b-8c20-c665018c84a2 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:24.440007 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c2acf-adb9-425b-8c20-c665018c84a2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:24.441884 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c2acf-adb9-425b-8c20-c665018c84a2 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:24.458859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe1b05-b48f-4335-be68-94478c1ffacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:24.463114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe1b05-b48f-4335-be68-94478c1ffacb None admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:24.463114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe1b05-b48f-4335-be68-94478c1ffacb None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:24.463114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe1b05-b48f-4335-be68-94478c1ffacb None admin] PATH_INFO: `/v3/domains/default/users/31dac45eb8764daf9a148a750fdc78a0/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:24.463114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe1b05-b48f-4335-be68-94478c1ffacb None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=31dac45eb8764daf9a148a750fdc78a0, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:24.464275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe1b05-b48f-4335-be68-94478c1ffacb None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:24.520083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 18/39] 10.222.0.94 () {60 vars in 1131 bytes} [Tue Jul 26 14:39:24 2022] POST /identity/v3/roles => generated 225 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:39:24.521039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c2acf-adb9-425b-8c20-c665018c84a2 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:24.566552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:24.682516 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c2acf-adb9-425b-8c20-c665018c84a2 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:24.750450 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57147f7e-93c2-4e67-b5df-d1cdb6d0f219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:24.751219 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57147f7e-93c2-4e67-b5df-d1cdb6d0f219 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:24.751609 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57147f7e-93c2-4e67-b5df-d1cdb6d0f219 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:24.751901 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57147f7e-93c2-4e67-b5df-d1cdb6d0f219 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:24.752698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57147f7e-93c2-4e67-b5df-d1cdb6d0f219 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:24.755521 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57147f7e-93c2-4e67-b5df-d1cdb6d0f219 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:24.839961 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 19/40] 10.222.0.94 () {60 vars in 1131 bytes} [Tue Jul 26 14:39:24 2022] POST /identity/v3/roles => generated 223 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:39:24.841343 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57147f7e-93c2-4e67-b5df-d1cdb6d0f219 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:24.888238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 22/41] 10.222.0.94 () {58 vars in 1361 bytes} [Tue Jul 26 14:39:24 2022] PUT /identity/v3/domains/default/users/31dac45eb8764daf9a148a750fdc78a0/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:39:24.889722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de381c05-762d-4829-bb64-e6f6b4600069 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:24.890006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de381c05-762d-4829-bb64-e6f6b4600069 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:24.890550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de381c05-762d-4829-bb64-e6f6b4600069 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:24.895515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 23/42] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:24.905305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd69264-aa02-4fbb-946c-65de97df099b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:24.905769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd69264-aa02-4fbb-946c-65de97df099b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:24.906116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd69264-aa02-4fbb-946c-65de97df099b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:24.940559 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-0bd69264-aa02-4fbb-946c-65de97df099b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:24.943508 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-0bd69264-aa02-4fbb-946c-65de97df099b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:25.010938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:25.046027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:25.077305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57147f7e-93c2-4e67-b5df-d1cdb6d0f219 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:25.109578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 24/43] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:39:25.145146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81780ebc-7f27-4ad0-837d-9e6d06f04167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:25.146653 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81780ebc-7f27-4ad0-837d-9e6d06f04167 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:25.148091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81780ebc-7f27-4ad0-837d-9e6d06f04167 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:25.149560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81780ebc-7f27-4ad0-837d-9e6d06f04167 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:25.151204 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81780ebc-7f27-4ad0-837d-9e6d06f04167 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:25.151857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd69264-aa02-4fbb-946c-65de97df099b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:25.153058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81780ebc-7f27-4ad0-837d-9e6d06f04167 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:25.278870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 20/44] 10.222.0.94 () {60 vars in 1141 bytes} [Tue Jul 26 14:39:24 2022] POST /identity/v3/projects => generated 301 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:39:25.317209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd69264-aa02-4fbb-946c-65de97df099b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:25.369926 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f050340-91da-43f3-916b-5ecb43fde477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:25.370943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f050340-91da-43f3-916b-5ecb43fde477 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:25.371436 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f050340-91da-43f3-916b-5ecb43fde477 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:25.372002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f050340-91da-43f3-916b-5ecb43fde477 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:25.373682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f050340-91da-43f3-916b-5ecb43fde477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:25.375656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f050340-91da-43f3-916b-5ecb43fde477 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:25.377680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 25/45] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:39:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:25.387059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81780ebc-7f27-4ad0-837d-9e6d06f04167 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:25.548975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81780ebc-7f27-4ad0-837d-9e6d06f04167 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:25.615805 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c14241-93b8-4f68-8ca0-ab06ad776d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:25.616724 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c14241-93b8-4f68-8ca0-ab06ad776d1e None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:25.617175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c14241-93b8-4f68-8ca0-ab06ad776d1e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:25.617456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c14241-93b8-4f68-8ca0-ab06ad776d1e None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:25.618347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c14241-93b8-4f68-8ca0-ab06ad776d1e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:25.620445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c14241-93b8-4f68-8ca0-ab06ad776d1e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:25.633814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:25.743507 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 21/46] 10.222.0.94 () {60 vars in 1141 bytes} [Tue Jul 26 14:39:25 2022] POST /identity/v3/projects => generated 312 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:39:27.138911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9c220-4779-4fef-81b3-8987b07500bd None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:27.139513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9c220-4779-4fef-81b3-8987b07500bd None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:27.139513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9c220-4779-4fef-81b3-8987b07500bd None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:27.141095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 26/47] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:27.148040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cff9f-96b7-4e8f-8198-b5c357cd072b None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:27.148891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cff9f-96b7-4e8f-8198-b5c357cd072b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:27.148891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cff9f-96b7-4e8f-8198-b5c357cd072b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:27.168718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ce1cff9f-96b7-4e8f-8198-b5c357cd072b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:27.169979 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ce1cff9f-96b7-4e8f-8198-b5c357cd072b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:27.258655 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 22/48] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:27.284714 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9c220-4779-4fef-81b3-8987b07500bd None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:27.325729 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:27.415600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9c220-4779-4fef-81b3-8987b07500bd None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:27.458110 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d951b-3ef2-48eb-aac4-732e5d37e724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:27.458551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d951b-3ef2-48eb-aac4-732e5d37e724 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:27.458832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d951b-3ef2-48eb-aac4-732e5d37e724 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:27.459103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d951b-3ef2-48eb-aac4-732e5d37e724 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:27.472544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d951b-3ef2-48eb-aac4-732e5d37e724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-a38d951b-3ef2-48eb-aac4-732e5d37e724 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:27.478865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:27.479946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:27.480757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:27.480757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:27.480757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:27.480757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:27.480757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:27.480757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:27.480757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:27.480757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:27.480757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:27.480757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:27.480757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:39:27.480757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:39:27.480757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 27/49] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:39:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:27.486680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1cff9f-96b7-4e8f-8198-b5c357cd072b None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:27.608938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1cff9f-96b7-4e8f-8198-b5c357cd072b None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:27.635145 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:27.653861 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781afa6-9792-4f3e-b979-7f128b0bc48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:27.654631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781afa6-9792-4f3e-b979-7f128b0bc48c None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:27.654976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781afa6-9792-4f3e-b979-7f128b0bc48c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:27.655306 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781afa6-9792-4f3e-b979-7f128b0bc48c None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:27.656180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781afa6-9792-4f3e-b979-7f128b0bc48c None admin] RBAC: Adding query filter params (name=admin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:27.656546 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781afa6-9792-4f3e-b979-7f128b0bc48c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:27.658046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781afa6-9792-4f3e-b979-7f128b0bc48c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:27.669484 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 23/50] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:27.675261 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d951b-3ef2-48eb-aac4-732e5d37e724 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:27.792249 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:27.803189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d951b-3ef2-48eb-aac4-732e5d37e724 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:27.845609 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7ba53-f58a-4711-842d-d9f73919cc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:27.846270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7ba53-f58a-4711-842d-d9f73919cc04 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:27.846558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7ba53-f58a-4711-842d-d9f73919cc04 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:27.846838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7ba53-f58a-4711-842d-d9f73919cc04 None admin] PATH_INFO: `/v3/users/31dac45eb8764daf9a148a750fdc78a0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:27.848140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7ba53-f58a-4711-842d-d9f73919cc04 None admin] RBAC: Authorizing `identity:get_user(user_id=31dac45eb8764daf9a148a750fdc78a0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:27.849665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7ba53-f58a-4711-842d-d9f73919cc04 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:27.852146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 28/51] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:27 2022] GET /identity/v3/users/31dac45eb8764daf9a148a750fdc78a0 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:27.858202 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781afa6-9792-4f3e-b979-7f128b0bc48c None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:28.045136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781afa6-9792-4f3e-b979-7f128b0bc48c None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:28.113301 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6102ff-a7e7-4f56-bc6a-c4cc8e4c001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:28.114437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6102ff-a7e7-4f56-bc6a-c4cc8e4c001e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:28.114955 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6102ff-a7e7-4f56-bc6a-c4cc8e4c001e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:28.115518 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6102ff-a7e7-4f56-bc6a-c4cc8e4c001e None admin] PATH_INFO: `/v3/domains/default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:28.118486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6102ff-a7e7-4f56-bc6a-c4cc8e4c001e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:28.121398 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6102ff-a7e7-4f56-bc6a-c4cc8e4c001e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:28.124579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 24/52] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:39:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:28.134795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7ba53-f58a-4711-842d-d9f73919cc04 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:28.291690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7ba53-f58a-4711-842d-d9f73919cc04 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:28.345601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f424847-f3ff-44fe-b901-223549a24b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:28.346259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f424847-f3ff-44fe-b901-223549a24b11 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:28.346610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f424847-f3ff-44fe-b901-223549a24b11 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:28.346963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f424847-f3ff-44fe-b901-223549a24b11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:28.347906 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f424847-f3ff-44fe-b901-223549a24b11 None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.domain.id=default, user.id=31dac45eb8764daf9a148a750fdc78a0) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:28.348287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f424847-f3ff-44fe-b901-223549a24b11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:28.350826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f424847-f3ff-44fe-b901-223549a24b11 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:28.358871 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 29/53] 10.222.0.94 () {56 vars in 1322 bytes} [Tue Jul 26 14:39:28 2022] GET /identity/v3/role_assignments?user.id=31dac45eb8764daf9a148a750fdc78a0&scope.domain.id=default&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 530 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:28.838081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdd316-4b3e-4e7b-ba3f-88bb7fcc3c8f None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:28.838819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdd316-4b3e-4e7b-ba3f-88bb7fcc3c8f None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:28.838819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdd316-4b3e-4e7b-ba3f-88bb7fcc3c8f None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:28.840470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 25/54] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:28.847300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cda27-c083-4bbc-8ad5-2ae75d910ae7 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:28.847639 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cda27-c083-4bbc-8ad5-2ae75d910ae7 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:28.847917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cda27-c083-4bbc-8ad5-2ae75d910ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:28.869606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-d29cda27-c083-4bbc-8ad5-2ae75d910ae7 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:28.870919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d29cda27-c083-4bbc-8ad5-2ae75d910ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:28.892322 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:28.954200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 30/55] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:28.996725 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdd316-4b3e-4e7b-ba3f-88bb7fcc3c8f None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:29.092164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:29.155171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdd316-4b3e-4e7b-ba3f-88bb7fcc3c8f None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:29.198179 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a23ee-5c32-48e4-8e5a-ed2686843125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:29.198903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a23ee-5c32-48e4-8e5a-ed2686843125 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:29.199268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a23ee-5c32-48e4-8e5a-ed2686843125 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:29.199661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a23ee-5c32-48e4-8e5a-ed2686843125 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:29.201229 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a23ee-5c32-48e4-8e5a-ed2686843125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:29.202784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a23ee-5c32-48e4-8e5a-ed2686843125 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:29.204683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 26/56] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:39:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:29.211217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cda27-c083-4bbc-8ad5-2ae75d910ae7 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:29.333984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cda27-c083-4bbc-8ad5-2ae75d910ae7 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:29.377458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307160a-1b24-4904-ad50-eaa30a89595b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:29.378017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307160a-1b24-4904-ad50-eaa30a89595b None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:29.378298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307160a-1b24-4904-ad50-eaa30a89595b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:29.378572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307160a-1b24-4904-ad50-eaa30a89595b None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:29.379284 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307160a-1b24-4904-ad50-eaa30a89595b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:29.380828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307160a-1b24-4904-ad50-eaa30a89595b None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:29.481189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 31/57] 10.222.0.94 () {60 vars in 1140 bytes} [Tue Jul 26 14:39:29 2022] POST /identity/v3/projects => generated 298 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:39:31.721081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15dbec-d57b-4b18-a6a3-7f0240235832 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:31.721877 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15dbec-d57b-4b18-a6a3-7f0240235832 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:31.721877 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15dbec-d57b-4b18-a6a3-7f0240235832 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:31.723154 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 27/58] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:39:31.729672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b4841-07a0-49be-8ab8-ff273c3dae99 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:31.730352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b4841-07a0-49be-8ab8-ff273c3dae99 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:31.730352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b4841-07a0-49be-8ab8-ff273c3dae99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:31.745148 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:31.751575 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-8c2b4841-07a0-49be-8ab8-ff273c3dae99 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:31.752973 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-8c2b4841-07a0-49be-8ab8-ff273c3dae99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:31.838065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 32/59] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:39:31.865069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15dbec-d57b-4b18-a6a3-7f0240235832 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:31.989102 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15dbec-d57b-4b18-a6a3-7f0240235832 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:32.020435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:32.042254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8a452-95b0-45bd-b2bf-a08ad1efeccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:32.042983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8a452-95b0-45bd-b2bf-a08ad1efeccb None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:32.043201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8a452-95b0-45bd-b2bf-a08ad1efeccb None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:32.043513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8a452-95b0-45bd-b2bf-a08ad1efeccb None admin] PATH_INFO: `/v3/domains/default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:32.044786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8a452-95b0-45bd-b2bf-a08ad1efeccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:32.046385 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8a452-95b0-45bd-b2bf-a08ad1efeccb None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:32.048315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 28/60] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:39:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:32.054773 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b4841-07a0-49be-8ab8-ff273c3dae99 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:32.180770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b4841-07a0-49be-8ab8-ff273c3dae99 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:32.230217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e33d07-2613-45c9-9e2d-d9c8a6688013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:32.230967 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e33d07-2613-45c9-9e2d-d9c8a6688013 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:32.231255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e33d07-2613-45c9-9e2d-d9c8a6688013 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:32.231566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e33d07-2613-45c9-9e2d-d9c8a6688013 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:32.232312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e33d07-2613-45c9-9e2d-d9c8a6688013 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:32.233847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e33d07-2613-45c9-9e2d-d9c8a6688013 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:32.247558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:32.426124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 33/61] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:39:32 2022] POST /identity/v3/users => generated 275 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:39:36.718894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5e185-a9b5-4413-96d5-87b69e02f1e1 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:36.719970 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5e185-a9b5-4413-96d5-87b69e02f1e1 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:36.720488 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5e185-a9b5-4413-96d5-87b69e02f1e1 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:36.722917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 29/62] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:36.729605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0bbf1-7009-461e-8055-8ae95f8e55a0 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:36.730113 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0bbf1-7009-461e-8055-8ae95f8e55a0 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:36.730208 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0bbf1-7009-461e-8055-8ae95f8e55a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:36.760948 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-e1c0bbf1-7009-461e-8055-8ae95f8e55a0 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:36.762442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e1c0bbf1-7009-461e-8055-8ae95f8e55a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:36.886449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 34/63] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:36.924515 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5e185-a9b5-4413-96d5-87b69e02f1e1 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:37.073876 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cbf86-2c6b-49eb-bc5c-f178205ae2b7 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:37.074560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cbf86-2c6b-49eb-bc5c-f178205ae2b7 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:37.075043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cbf86-2c6b-49eb-bc5c-f178205ae2b7 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:37.077230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 35/64] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:37.085081 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923451ee-9fc1-4ae5-9408-acaab28f52dd None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:37.085702 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923451ee-9fc1-4ae5-9408-acaab28f52dd None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:37.086204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923451ee-9fc1-4ae5-9408-acaab28f52dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:37.095243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5e185-a9b5-4413-96d5-87b69e02f1e1 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:37.117839 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-923451ee-9fc1-4ae5-9408-acaab28f52dd None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:37.120645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-923451ee-9fc1-4ae5-9408-acaab28f52dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:37.146802 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70385f91-3f8c-432a-b64e-a75ebef07cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:37.148632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70385f91-3f8c-432a-b64e-a75ebef07cf1 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:37.149153 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70385f91-3f8c-432a-b64e-a75ebef07cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:37.149720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70385f91-3f8c-432a-b64e-a75ebef07cf1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:37.166780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70385f91-3f8c-432a-b64e-a75ebef07cf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-70385f91-3f8c-432a-b64e-a75ebef07cf1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:37.174656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:37.175635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:37.176663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:37.176663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:37.176663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:37.176663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:37.176663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:37.176663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:37.176663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:37.176663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:37.176663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:37.176663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:37.176663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:39:37.176663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:39:37.178547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 30/65] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:37.182593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc469857-4d97-4ecb-9621-c0df88b41be8 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:37.183086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc469857-4d97-4ecb-9621-c0df88b41be8 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:37.186553 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc469857-4d97-4ecb-9621-c0df88b41be8 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:37.189832 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 31/66] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:37 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:37.191155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc469857-4d97-4ecb-9621-c0df88b41be8 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:37.263195 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:37.294173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 36/67] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:37.297173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a0bc9-5570-439c-8700-e3ce925f3a8e None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:37.298095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a0bc9-5570-439c-8700-e3ce925f3a8e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:37.298937 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a0bc9-5570-439c-8700-e3ce925f3a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:37.330661 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-708a0bc9-5570-439c-8700-e3ce925f3a8e None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:37.332819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-708a0bc9-5570-439c-8700-e3ce925f3a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:37.343141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc469857-4d97-4ecb-9621-c0df88b41be8 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:37.390075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f28573-8732-4c0b-87ec-1d6d794f040e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:37.390732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f28573-8732-4c0b-87ec-1d6d794f040e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:37.391093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f28573-8732-4c0b-87ec-1d6d794f040e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:37.391307 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f28573-8732-4c0b-87ec-1d6d794f040e None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:37.392102 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f28573-8732-4c0b-87ec-1d6d794f040e None admin] RBAC: Adding query filter params (name=member) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:37.392417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f28573-8732-4c0b-87ec-1d6d794f040e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:37.393903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f28573-8732-4c0b-87ec-1d6d794f040e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:37.404941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 32/68] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:39:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:37.406474 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9a0a1-f3b6-45d4-b9d5-410a537fe71f None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:37.406795 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9a0a1-f3b6-45d4-b9d5-410a537fe71f None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:37.407396 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9a0a1-f3b6-45d4-b9d5-410a537fe71f None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:37.408920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 33/69] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:37.409565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9a0a1-f3b6-45d4-b9d5-410a537fe71f None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:37.425659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:37.453682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 37/70] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:37.455054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f45d4-9d15-4dc9-a502-ac5d81f965dc None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:37.455341 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f45d4-9d15-4dc9-a502-ac5d81f965dc None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:37.455682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f45d4-9d15-4dc9-a502-ac5d81f965dc None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:37.457416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 38/71] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:37.457926 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f45d4-9d15-4dc9-a502-ac5d81f965dc None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:37.550203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9a0a1-f3b6-45d4-b9d5-410a537fe71f None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:37.562086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:37.597012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f45d4-9d15-4dc9-a502-ac5d81f965dc None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:37.597706 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb6c18-8e7c-4d80-89d3-66c5931c23ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:37.598379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb6c18-8e7c-4d80-89d3-66c5931c23ee None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:37.598674 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb6c18-8e7c-4d80-89d3-66c5931c23ee None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:37.598947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb6c18-8e7c-4d80-89d3-66c5931c23ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:37.614177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb6c18-8e7c-4d80-89d3-66c5931c23ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-b6eb6c18-8e7c-4d80-89d3-66c5931c23ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:37.621265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:37.622489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:37.623325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:37.623325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:37.623325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:37.623325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:37.623325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:37.623325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:37.623325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:37.623325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:37.623325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:37.623325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:37.623325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:39:37.623325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:39:37.623325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 34/72] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:39:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:37.624055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b7e65-8e3e-429d-86c4-596bcfd3e07d None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:37.624221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b7e65-8e3e-429d-86c4-596bcfd3e07d None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:37.624524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b7e65-8e3e-429d-86c4-596bcfd3e07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:37.650351 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625163f-1bf2-413c-b725-af3de75ba604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:37.651449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ae6b7e65-8e3e-429d-86c4-596bcfd3e07d None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:37.651661 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625163f-1bf2-413c-b725-af3de75ba604 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:37.651976 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625163f-1bf2-413c-b725-af3de75ba604 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:37.652457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625163f-1bf2-413c-b725-af3de75ba604 None admin] PATH_INFO: `/v3/users/843d8bc8740f48aaa93c0f79dc9687d0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:37.653497 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ae6b7e65-8e3e-429d-86c4-596bcfd3e07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:37.671227 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625163f-1bf2-413c-b725-af3de75ba604 None admin] RBAC: Authorizing `identity:get_user(user_id=843d8bc8740f48aaa93c0f79dc9687d0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:37.673157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625163f-1bf2-413c-b725-af3de75ba604 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:37.675217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 39/73] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:37 2022] GET /identity/v3/users/843d8bc8740f48aaa93c0f79dc9687d0 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:37.676381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6419db5-9708-4e86-a7f8-a9fb4ae7c672 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:37.676730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6419db5-9708-4e86-a7f8-a9fb4ae7c672 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:37.677009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6419db5-9708-4e86-a7f8-a9fb4ae7c672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:37.702744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c6419db5-9708-4e86-a7f8-a9fb4ae7c672 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:37.704023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c6419db5-9708-4e86-a7f8-a9fb4ae7c672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:37.750397 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:37.757227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 35/74] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:37.758167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b7e65-8e3e-429d-86c4-596bcfd3e07d None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:37.799260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 40/75] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:37.800601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6419db5-9708-4e86-a7f8-a9fb4ae7c672 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:37.908533 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:37.914733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b7e65-8e3e-429d-86c4-596bcfd3e07d None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:37.963182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74563420-4ded-4bd7-8f62-3be559d26b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:37.963906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74563420-4ded-4bd7-8f62-3be559d26b3f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:37.964196 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74563420-4ded-4bd7-8f62-3be559d26b3f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:37.964480 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74563420-4ded-4bd7-8f62-3be559d26b3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:37.978964 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74563420-4ded-4bd7-8f62-3be559d26b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:37.983901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6419db5-9708-4e86-a7f8-a9fb4ae7c672 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-74563420-4ded-4bd7-8f62-3be559d26b3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:37.984918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:37.986718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:37.988439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:37.988439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:37.988439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:37.988439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:37.988439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:37.988439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:37.988439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:37.988439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:37.988439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:37.988439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:37.988439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:39:37.988439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:39:37.988439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 36/76] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:39:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:37.992079 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74563420-4ded-4bd7-8f62-3be559d26b3f None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:38.046448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf883dc1-7b2b-408d-b36f-ecd25aa4452d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:38.047092 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf883dc1-7b2b-408d-b36f-ecd25aa4452d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:38.047385 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf883dc1-7b2b-408d-b36f-ecd25aa4452d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:38.047693 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf883dc1-7b2b-408d-b36f-ecd25aa4452d None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:38.048442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf883dc1-7b2b-408d-b36f-ecd25aa4452d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:38.048752 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf883dc1-7b2b-408d-b36f-ecd25aa4452d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:38.050176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf883dc1-7b2b-408d-b36f-ecd25aa4452d None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:38.060197 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 41/77] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:39:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:38.061161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf883dc1-7b2b-408d-b36f-ecd25aa4452d None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:38.114233 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:38.152183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74563420-4ded-4bd7-8f62-3be559d26b3f None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:38.205346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524cebe-9ccb-46c6-86bb-ccc338e2a162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:38.206262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524cebe-9ccb-46c6-86bb-ccc338e2a162 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:38.206717 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524cebe-9ccb-46c6-86bb-ccc338e2a162 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:38.207186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524cebe-9ccb-46c6-86bb-ccc338e2a162 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:38.221647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf883dc1-7b2b-408d-b36f-ecd25aa4452d None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:38.261268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:38.281018 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163030e4-c387-41cc-bc17-a1f61c5eea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:38.282206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163030e4-c387-41cc-bc17-a1f61c5eea90 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:38.282688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163030e4-c387-41cc-bc17-a1f61c5eea90 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:38.283157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163030e4-c387-41cc-bc17-a1f61c5eea90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:38.304210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163030e4-c387-41cc-bc17-a1f61c5eea90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-163030e4-c387-41cc-bc17-a1f61c5eea90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:38.313924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:38.315608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:38.316544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:38.316544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:38.316544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:38.316544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:38.316544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:38.316544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:38.316544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:38.316544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:38.316544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:38.316544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:38.316544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:39:38.316544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:39:38.316544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 42/78] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:38.317365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163030e4-c387-41cc-bc17-a1f61c5eea90 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:38.321869 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524cebe-9ccb-46c6-86bb-ccc338e2a162 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:38.323875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524cebe-9ccb-46c6-86bb-ccc338e2a162 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:38.454120 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:38.467203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 37/79] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:39:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:39:38.467203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524cebe-9ccb-46c6-86bb-ccc338e2a162 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:38.467692 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163030e4-c387-41cc-bc17-a1f61c5eea90 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:38.510767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd832249-81a7-4415-8a83-dbf6bcdd446c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:38.511437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd832249-81a7-4415-8a83-dbf6bcdd446c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:38.511758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd832249-81a7-4415-8a83-dbf6bcdd446c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:38.512038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd832249-81a7-4415-8a83-dbf6bcdd446c None admin] PATH_INFO: `/v3/domains/default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:38.513444 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd832249-81a7-4415-8a83-dbf6bcdd446c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:38.514972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd832249-81a7-4415-8a83-dbf6bcdd446c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:38.516853 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 43/80] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:39:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:38.517678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd832249-81a7-4415-8a83-dbf6bcdd446c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:38.570746 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:38.638903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524cebe-9ccb-46c6-86bb-ccc338e2a162 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:38.641582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd832249-81a7-4415-8a83-dbf6bcdd446c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:38.684277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012ea7e-64ab-4870-ac11-89d952afedbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:38.685034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012ea7e-64ab-4870-ac11-89d952afedbc None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:38.685034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012ea7e-64ab-4870-ac11-89d952afedbc None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:38.685034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012ea7e-64ab-4870-ac11-89d952afedbc None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:38.686358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012ea7e-64ab-4870-ac11-89d952afedbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:38.686657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012ea7e-64ab-4870-ac11-89d952afedbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:38.688116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012ea7e-64ab-4870-ac11-89d952afedbc None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:38.691182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882f2ce-dd0e-4e7b-bc81-0220778d49cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:38.692321 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882f2ce-dd0e-4e7b-bc81-0220778d49cf None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:38.692716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882f2ce-dd0e-4e7b-bc81-0220778d49cf None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:38.693023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882f2ce-dd0e-4e7b-bc81-0220778d49cf None admin] PATH_INFO: `/v3/users/843d8bc8740f48aaa93c0f79dc9687d0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:38.694945 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882f2ce-dd0e-4e7b-bc81-0220778d49cf None admin] RBAC: Authorizing `identity:get_user(user_id=843d8bc8740f48aaa93c0f79dc9687d0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:38.697509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882f2ce-dd0e-4e7b-bc81-0220778d49cf None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:38.698215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 38/81] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:38.699108 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012ea7e-64ab-4870-ac11-89d952afedbc None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:38.706879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 44/82] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:38 2022] GET /identity/v3/users/843d8bc8740f48aaa93c0f79dc9687d0 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:38.706879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882f2ce-dd0e-4e7b-bc81-0220778d49cf None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:38.756452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:38.854383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012ea7e-64ab-4870-ac11-89d952afedbc None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:38.863946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882f2ce-dd0e-4e7b-bc81-0220778d49cf None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:38.881953 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:38.909768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464e6aa-ac51-47c9-9708-be3f410476ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:38.910884 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464e6aa-ac51-47c9-9708-be3f410476ec None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:38.911395 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464e6aa-ac51-47c9-9708-be3f410476ec None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:38.911950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464e6aa-ac51-47c9-9708-be3f410476ec None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:38.914293 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464e6aa-ac51-47c9-9708-be3f410476ec None admin] RBAC: Adding query filter params (name=member) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:38.914848 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464e6aa-ac51-47c9-9708-be3f410476ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:38.915004 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20147e36-98e9-4a6a-a774-6ecdcb84c472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:38.915635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20147e36-98e9-4a6a-a774-6ecdcb84c472 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:38.915918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20147e36-98e9-4a6a-a774-6ecdcb84c472 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:38.916191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20147e36-98e9-4a6a-a774-6ecdcb84c472 None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:38.917420 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20147e36-98e9-4a6a-a774-6ecdcb84c472 None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:38.917531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464e6aa-ac51-47c9-9708-be3f410476ec None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:38.918874 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20147e36-98e9-4a6a-a774-6ecdcb84c472 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:38.921071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 45/83] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:39:38 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:38.921508 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20147e36-98e9-4a6a-a774-6ecdcb84c472 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:38.941792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 39/84] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:39:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:38.941792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464e6aa-ac51-47c9-9708-be3f410476ec None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:39.072659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:39.079555 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464e6aa-ac51-47c9-9708-be3f410476ec None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:39.084898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20147e36-98e9-4a6a-a774-6ecdcb84c472 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:39.128238 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283b14f-6b26-40ee-973a-6c9b34e1270a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:39.129026 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283b14f-6b26-40ee-973a-6c9b34e1270a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:39.129324 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283b14f-6b26-40ee-973a-6c9b34e1270a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:39.129579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283b14f-6b26-40ee-973a-6c9b34e1270a None admin] PATH_INFO: `/v3/users/31dac45eb8764daf9a148a750fdc78a0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:39.132205 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283b14f-6b26-40ee-973a-6c9b34e1270a None admin] RBAC: Authorizing `identity:get_user(user_id=31dac45eb8764daf9a148a750fdc78a0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:39.133734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283b14f-6b26-40ee-973a-6c9b34e1270a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:39.135491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 40/85] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:38 2022] GET /identity/v3/users/31dac45eb8764daf9a148a750fdc78a0 => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:39.136321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283b14f-6b26-40ee-973a-6c9b34e1270a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:39.142121 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe79f84-bf9f-476e-9fe4-cb61f3c5baf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:39.143386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe79f84-bf9f-476e-9fe4-cb61f3c5baf7 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:39.143785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe79f84-bf9f-476e-9fe4-cb61f3c5baf7 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:39.144092 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe79f84-bf9f-476e-9fe4-cb61f3c5baf7 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:39.145014 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe79f84-bf9f-476e-9fe4-cb61f3c5baf7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:39.147189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe79f84-bf9f-476e-9fe4-cb61f3c5baf7 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:39.207391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:39.303509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 46/86] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:39:38 2022] POST /identity/v3/users => generated 289 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:39:39.303509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe79f84-bf9f-476e-9fe4-cb61f3c5baf7 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:39.304103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283b14f-6b26-40ee-973a-6c9b34e1270a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:39.358567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d57d3-191f-4041-a80b-28fe55c7dc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:39.359520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d57d3-191f-4041-a80b-28fe55c7dc23 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:39.360002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d57d3-191f-4041-a80b-28fe55c7dc23 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:39.360473 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d57d3-191f-4041-a80b-28fe55c7dc23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:39.496144 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d57d3-191f-4041-a80b-28fe55c7dc23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:39.499136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d57d3-191f-4041-a80b-28fe55c7dc23 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:39.500516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe79f84-bf9f-476e-9fe4-cb61f3c5baf7 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:39.568391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb82fe-2bbd-454f-9504-f78538b895cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:39.569886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb82fe-2bbd-454f-9504-f78538b895cc None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:39.570363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb82fe-2bbd-454f-9504-f78538b895cc None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:39.570838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb82fe-2bbd-454f-9504-f78538b895cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:39.572362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb82fe-2bbd-454f-9504-f78538b895cc None admin] RBAC: Adding query filter params (role.id=6a924f1d619643b3be92bab6fddb3eb2, scope.project.id=d0c721e2529541e5966d3c21426e447a, user.id=843d8bc8740f48aaa93c0f79dc9687d0) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:39.572881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb82fe-2bbd-454f-9504-f78538b895cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:39.575514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb82fe-2bbd-454f-9504-f78538b895cc None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:39.584238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 47/87] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:39:39 2022] GET /identity/v3/role_assignments?user.id=843d8bc8740f48aaa93c0f79dc9687d0&scope.project.id=d0c721e2529541e5966d3c21426e447a&role.id=6a924f1d619643b3be92bab6fddb3eb2 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:39.586449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb82fe-2bbd-454f-9504-f78538b895cc None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:39.611305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:39.623177 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:39.700274 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 41/88] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:39:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:39:39.701609 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d57d3-191f-4041-a80b-28fe55c7dc23 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:39.803107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb82fe-2bbd-454f-9504-f78538b895cc None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:39.851894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d57d3-191f-4041-a80b-28fe55c7dc23 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:39.889461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558b6db-0f7b-4500-8cb1-4952a8cafe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:39.890496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558b6db-0f7b-4500-8cb1-4952a8cafe47 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:39.890994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558b6db-0f7b-4500-8cb1-4952a8cafe47 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:39.891510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558b6db-0f7b-4500-8cb1-4952a8cafe47 None admin] PATH_INFO: `/v3/users/843d8bc8740f48aaa93c0f79dc9687d0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:39.893824 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558b6db-0f7b-4500-8cb1-4952a8cafe47 None admin] RBAC: Authorizing `identity:get_user(user_id=843d8bc8740f48aaa93c0f79dc9687d0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:39.897407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558b6db-0f7b-4500-8cb1-4952a8cafe47 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:39.901996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 48/89] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:39 2022] GET /identity/v3/users/843d8bc8740f48aaa93c0f79dc9687d0 => generated 275 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:39.903566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558b6db-0f7b-4500-8cb1-4952a8cafe47 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:39.931353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d5370-3bfb-4433-b8a0-d122d1af1b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:39.937626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d5370-3bfb-4433-b8a0-d122d1af1b37 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:39.938534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d5370-3bfb-4433-b8a0-d122d1af1b37 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:39.939317 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d5370-3bfb-4433-b8a0-d122d1af1b37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:40.016098 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:40.103765 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:40.174659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558b6db-0f7b-4500-8cb1-4952a8cafe47 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:40.197642 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d5370-3bfb-4433-b8a0-d122d1af1b37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:40.201080 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d5370-3bfb-4433-b8a0-d122d1af1b37 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:40.317664 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ce483-0cf9-4552-83ca-ea55221841c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:40.318760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ce483-0cf9-4552-83ca-ea55221841c9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:40.319259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ce483-0cf9-4552-83ca-ea55221841c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:40.319768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ce483-0cf9-4552-83ca-ea55221841c9 None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:40.326514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ce483-0cf9-4552-83ca-ea55221841c9 None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:40.329365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ce483-0cf9-4552-83ca-ea55221841c9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:40.344098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 49/90] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:39:39 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:40.345564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ce483-0cf9-4552-83ca-ea55221841c9 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:40.526331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 42/91] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:39:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 825 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:39:40.527920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d5370-3bfb-4433-b8a0-d122d1af1b37 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:40.614312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ce483-0cf9-4552-83ca-ea55221841c9 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:40.700938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:40.724222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93d75b-9fc8-47fe-96c8-7db0fa29cc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:40.725975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93d75b-9fc8-47fe-96c8-7db0fa29cc34 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:40.726793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93d75b-9fc8-47fe-96c8-7db0fa29cc34 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:40.727619 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93d75b-9fc8-47fe-96c8-7db0fa29cc34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:40.788833 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:40.838493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d5370-3bfb-4433-b8a0-d122d1af1b37 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:40.921921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4ba5f-46fa-4724-ac61-f12e1f6eb5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:40.923165 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4ba5f-46fa-4724-ac61-f12e1f6eb5f4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:40.923726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4ba5f-46fa-4724-ac61-f12e1f6eb5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:40.924233 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4ba5f-46fa-4724-ac61-f12e1f6eb5f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:40.925830 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4ba5f-46fa-4724-ac61-f12e1f6eb5f4 None admin] RBAC: Adding query filter params (role.id=eff5cca513be4ed5a9b001c0559fb476, scope.project.id=d0c721e2529541e5966d3c21426e447a, user.id=843d8bc8740f48aaa93c0f79dc9687d0) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:40.926406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4ba5f-46fa-4724-ac61-f12e1f6eb5f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:40.930184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4ba5f-46fa-4724-ac61-f12e1f6eb5f4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:40.955649 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 43/92] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:39:40 2022] GET /identity/v3/role_assignments?user.id=843d8bc8740f48aaa93c0f79dc9687d0&scope.project.id=d0c721e2529541e5966d3c21426e447a&role.id=eff5cca513be4ed5a9b001c0559fb476 => generated 259 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:40.957265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4ba5f-46fa-4724-ac61-f12e1f6eb5f4 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:40.987499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93d75b-9fc8-47fe-96c8-7db0fa29cc34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:40.990220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93d75b-9fc8-47fe-96c8-7db0fa29cc34 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:41.320530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4ba5f-46fa-4724-ac61-f12e1f6eb5f4 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:41.362941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 50/93] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:39:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1016 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:39:41.369429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93d75b-9fc8-47fe-96c8-7db0fa29cc34 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:41.384096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:41.472779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784b286-d505-4bc3-ad76-4168b22992db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:41.474345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784b286-d505-4bc3-ad76-4168b22992db None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:41.476527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784b286-d505-4bc3-ad76-4168b22992db None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:41.477348 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784b286-d505-4bc3-ad76-4168b22992db None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:41.480140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784b286-d505-4bc3-ad76-4168b22992db None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:41.483492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784b286-d505-4bc3-ad76-4168b22992db None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:41.487244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 44/94] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:39:40 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 531 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:41.495052 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784b286-d505-4bc3-ad76-4168b22992db None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:41.532045 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:41.664122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93d75b-9fc8-47fe-96c8-7db0fa29cc34 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:41.730820 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d985e-33f4-4637-836a-7189a4b035fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:41.732459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d985e-33f4-4637-836a-7189a4b035fe None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:41.733438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d985e-33f4-4637-836a-7189a4b035fe None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:41.734179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d985e-33f4-4637-836a-7189a4b035fe None admin] PATH_INFO: `/v3/projects/66ad66f40fde439f881153a21f8c2b44` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:41.737158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d985e-33f4-4637-836a-7189a4b035fe None admin] RBAC: Authorizing `identity:get_project(project_id=66ad66f40fde439f881153a21f8c2b44)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:41.741096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d985e-33f4-4637-836a-7189a4b035fe None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:41.745181 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 51/95] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:39:41 2022] GET /identity/v3/projects/66ad66f40fde439f881153a21f8c2b44 => generated 312 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:41.761220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d985e-33f4-4637-836a-7189a4b035fe None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:41.846170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784b286-d505-4bc3-ad76-4168b22992db None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:41.937666 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8893c-706d-4a14-a1ff-dace73757b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:41.939024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8893c-706d-4a14-a1ff-dace73757b9e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:41.939787 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8893c-706d-4a14-a1ff-dace73757b9e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:41.940093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8893c-706d-4a14-a1ff-dace73757b9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:41.941726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8893c-706d-4a14-a1ff-dace73757b9e None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=d0c721e2529541e5966d3c21426e447a, user.id=31dac45eb8764daf9a148a750fdc78a0) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:41.942306 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8893c-706d-4a14-a1ff-dace73757b9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:41.944999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8893c-706d-4a14-a1ff-dace73757b9e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:41.969213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 45/96] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:39:41 2022] GET /identity/v3/role_assignments?user.id=31dac45eb8764daf9a148a750fdc78a0&scope.project.id=d0c721e2529541e5966d3c21426e447a&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 259 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:42.013687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d985e-33f4-4637-836a-7189a4b035fe None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:42.093261 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:42.147199 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af765fd-e1f7-44d8-8f24-87e5016c93bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:42.149062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af765fd-e1f7-44d8-8f24-87e5016c93bd None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:42.149863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af765fd-e1f7-44d8-8f24-87e5016c93bd None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:42.150630 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af765fd-e1f7-44d8-8f24-87e5016c93bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:42.152566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af765fd-e1f7-44d8-8f24-87e5016c93bd None admin] RBAC: Adding query filter params (role.id=6a924f1d619643b3be92bab6fddb3eb2, scope.project.id=66ad66f40fde439f881153a21f8c2b44, user.id=843d8bc8740f48aaa93c0f79dc9687d0) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:42.153457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af765fd-e1f7-44d8-8f24-87e5016c93bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:42.156659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af765fd-e1f7-44d8-8f24-87e5016c93bd None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:42.173392 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 52/97] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:39:41 2022] GET /identity/v3/role_assignments?user.id=843d8bc8740f48aaa93c0f79dc9687d0&scope.project.id=66ad66f40fde439f881153a21f8c2b44&role.id=6a924f1d619643b3be92bab6fddb3eb2 => generated 259 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:43.564714 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ab431-4c4a-4344-b9ac-0bd3ada32e9e None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:43.566356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ab431-4c4a-4344-b9ac-0bd3ada32e9e None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:43.567175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ab431-4c4a-4344-b9ac-0bd3ada32e9e None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:43.570906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 46/98] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:43.580100 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf57f8-8369-4d9f-b165-87c25e624f8f None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:43.581645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf57f8-8369-4d9f-b165-87c25e624f8f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:43.582399 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf57f8-8369-4d9f-b165-87c25e624f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:43.621182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-05bf57f8-8369-4d9f-b165-87c25e624f8f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:43.623690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-05bf57f8-8369-4d9f-b165-87c25e624f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:43.793551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 53/99] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:43.834371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ab431-4c4a-4344-b9ac-0bd3ada32e9e None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:43.961883 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:44.025288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033152f-71e9-474e-831a-5a6187106ced None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:44.025805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033152f-71e9-474e-831a-5a6187106ced None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:44.026569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033152f-71e9-474e-831a-5a6187106ced None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:44.029357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 54/100] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:44.065657 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4751372-58e4-40b2-8c8a-123d8d5a1e35 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:44.066119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4751372-58e4-40b2-8c8a-123d8d5a1e35 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:44.066620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4751372-58e4-40b2-8c8a-123d8d5a1e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:44.118773 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ab431-4c4a-4344-b9ac-0bd3ada32e9e None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:44.127853 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c4751372-58e4-40b2-8c8a-123d8d5a1e35 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:44.136128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c4751372-58e4-40b2-8c8a-123d8d5a1e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:44.265774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c409a4a-db48-41a6-ac4b-6cca80d19115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:44.267255 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c409a4a-db48-41a6-ac4b-6cca80d19115 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:44.267968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c409a4a-db48-41a6-ac4b-6cca80d19115 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:44.268658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c409a4a-db48-41a6-ac4b-6cca80d19115 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:44.271099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c409a4a-db48-41a6-ac4b-6cca80d19115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:44.283501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c409a4a-db48-41a6-ac4b-6cca80d19115 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:44.288038 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 47/101] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:39:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:44.309042 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c409a4a-db48-41a6-ac4b-6cca80d19115 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:44.363293 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 55/102] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:39:44.431896 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4751372-58e4-40b2-8c8a-123d8d5a1e35 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:44.576051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:44.695961 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c409a4a-db48-41a6-ac4b-6cca80d19115 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:44.728676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:44.772062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4751372-58e4-40b2-8c8a-123d8d5a1e35 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:44.794343 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17b73e-8814-4417-8ada-a8d7fbea234e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:44.795516 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17b73e-8814-4417-8ada-a8d7fbea234e None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:44.796029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17b73e-8814-4417-8ada-a8d7fbea234e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:44.796528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17b73e-8814-4417-8ada-a8d7fbea234e None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:44.797841 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17b73e-8814-4417-8ada-a8d7fbea234e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:44.800797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17b73e-8814-4417-8ada-a8d7fbea234e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:44.884844 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72326bbd-0a33-43cc-85c6-a5ba9a4fd889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:44.886010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72326bbd-0a33-43cc-85c6-a5ba9a4fd889 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:44.886505 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72326bbd-0a33-43cc-85c6-a5ba9a4fd889 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:44.886985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72326bbd-0a33-43cc-85c6-a5ba9a4fd889 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:44.908838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72326bbd-0a33-43cc-85c6-a5ba9a4fd889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-72326bbd-0a33-43cc-85c6-a5ba9a4fd889 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:44.920040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:44.921735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:44.923242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:44.923242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:44.923242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:44.923242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:44.923242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:44.923242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:44.923242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:44.923242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:44.923242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:44.923242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:44.923242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:39:44.923242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:39:44.923242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 56/103] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 492 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:39:44.924710 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a07a4-ec26-454c-adca-304314fd68bf None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:44.925456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a07a4-ec26-454c-adca-304314fd68bf None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:44.926098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a07a4-ec26-454c-adca-304314fd68bf None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:44.929828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 57/104] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:44.936485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a07a4-ec26-454c-adca-304314fd68bf None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:44.986892 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 48/105] 10.222.0.94 () {60 vars in 1141 bytes} [Tue Jul 26 14:39:44 2022] POST /identity/v3/projects => generated 302 bytes in 683 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:39:44.989669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b47ba-d328-49b8-94da-d4fb9fedddec None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:44.997271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b47ba-d328-49b8-94da-d4fb9fedddec None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:44.998303 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b47ba-d328-49b8-94da-d4fb9fedddec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:45.055212 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-766b47ba-d328-49b8-94da-d4fb9fedddec None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:45.057798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-766b47ba-d328-49b8-94da-d4fb9fedddec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:45.187446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a07a4-ec26-454c-adca-304314fd68bf None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:45.200092 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:45.339784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 49/106] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:45.352171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcc740-8925-4963-b24b-95f81c881128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:45.353853 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcc740-8925-4963-b24b-95f81c881128 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:45.354697 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcc740-8925-4963-b24b-95f81c881128 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:45.355486 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcc740-8925-4963-b24b-95f81c881128 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:45.357184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcc740-8925-4963-b24b-95f81c881128 None admin] RBAC: Adding query filter params (name=member) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:45.358051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcc740-8925-4963-b24b-95f81c881128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:45.361066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcc740-8925-4963-b24b-95f81c881128 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:45.389381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 58/107] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:39:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:45.398939 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b47ba-d328-49b8-94da-d4fb9fedddec None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:45.415009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcc740-8925-4963-b24b-95f81c881128 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:45.654387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcc740-8925-4963-b24b-95f81c881128 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:45.699284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:45.715690 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b47ba-d328-49b8-94da-d4fb9fedddec None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:45.745533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69154e-0350-45dd-93f2-515eea740ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:45.746870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69154e-0350-45dd-93f2-515eea740ab1 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:45.747554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69154e-0350-45dd-93f2-515eea740ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:45.748049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69154e-0350-45dd-93f2-515eea740ab1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:45.770916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69154e-0350-45dd-93f2-515eea740ab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-bb69154e-0350-45dd-93f2-515eea740ab1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:45.784223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:45.788358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:45.789827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:45.789827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:45.789827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:45.789827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:45.789827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:45.789827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:45.789827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:45.789827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:45.789827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:45.789827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:45.789827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:39:45.789827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:39:45.789827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 59/108] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:45.789827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43b067-5a76-4bb3-a29b-2bf58aa137cc None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:45.791226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43b067-5a76-4bb3-a29b-2bf58aa137cc None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:45.791226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43b067-5a76-4bb3-a29b-2bf58aa137cc None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:45.795532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 60/109] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:45.801737 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43b067-5a76-4bb3-a29b-2bf58aa137cc None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:45.864696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79526500-d9e1-4060-9de1-ed95ad08b569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:45.866867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79526500-d9e1-4060-9de1-ed95ad08b569 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:45.867655 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79526500-d9e1-4060-9de1-ed95ad08b569 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:45.868453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79526500-d9e1-4060-9de1-ed95ad08b569 None admin] PATH_INFO: `/v3/users/843d8bc8740f48aaa93c0f79dc9687d0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:45.873092 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:45.875388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79526500-d9e1-4060-9de1-ed95ad08b569 None admin] RBAC: Authorizing `identity:get_user(user_id=843d8bc8740f48aaa93c0f79dc9687d0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:45.878260 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79526500-d9e1-4060-9de1-ed95ad08b569 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:45.882031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 50/110] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:45 2022] GET /identity/v3/users/843d8bc8740f48aaa93c0f79dc9687d0 => generated 275 bytes in 484 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:45.889077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f9c4c-ffad-4d75-b10b-36edc89718a4 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:45.894787 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f9c4c-ffad-4d75-b10b-36edc89718a4 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:45.894787 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f9c4c-ffad-4d75-b10b-36edc89718a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:45.945087 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-dc6f9c4c-ffad-4d75-b10b-36edc89718a4 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:45.946671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-dc6f9c4c-ffad-4d75-b10b-36edc89718a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:46.073571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43b067-5a76-4bb3-a29b-2bf58aa137cc None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:46.121308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 51/111] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:46.122241 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f9c4c-ffad-4d75-b10b-36edc89718a4 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:46.166987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587239eb-4a91-4dfd-a18f-2bd3061acb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:46.168111 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587239eb-4a91-4dfd-a18f-2bd3061acb0b None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:46.168692 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587239eb-4a91-4dfd-a18f-2bd3061acb0b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:46.169187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587239eb-4a91-4dfd-a18f-2bd3061acb0b None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:46.170639 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587239eb-4a91-4dfd-a18f-2bd3061acb0b None admin] RBAC: Adding query filter params (name=reader) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:46.171191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587239eb-4a91-4dfd-a18f-2bd3061acb0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:46.174166 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587239eb-4a91-4dfd-a18f-2bd3061acb0b None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:46.205834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 61/112] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:39:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:46.214063 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587239eb-4a91-4dfd-a18f-2bd3061acb0b None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:46.450533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:46.470115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:46.475740 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587239eb-4a91-4dfd-a18f-2bd3061acb0b None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:46.488681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f9c4c-ffad-4d75-b10b-36edc89718a4 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:46.526709 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e69912-6b09-48a8-9836-ec84bb901e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:46.527855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e69912-6b09-48a8-9836-ec84bb901e4e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:46.528341 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e69912-6b09-48a8-9836-ec84bb901e4e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:46.528767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e69912-6b09-48a8-9836-ec84bb901e4e None admin] PATH_INFO: `/v3/users/84fbc773412c4d25a3aedf5914c636fc` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:46.548046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e69912-6b09-48a8-9836-ec84bb901e4e None admin] RBAC: Authorizing `identity:get_user(user_id=84fbc773412c4d25a3aedf5914c636fc)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:46.549837 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e69912-6b09-48a8-9836-ec84bb901e4e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:46.555912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 62/113] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:46 2022] GET /identity/v3/users/84fbc773412c4d25a3aedf5914c636fc => generated 289 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:46.557352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e69912-6b09-48a8-9836-ec84bb901e4e None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:46.580671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4b6b4-f9a8-4aca-9fac-22be278faa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:46.582126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b6b4-f9a8-4aca-9fac-22be278faa57 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:46.582844 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b6b4-f9a8-4aca-9fac-22be278faa57 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:46.584232 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b6b4-f9a8-4aca-9fac-22be278faa57 None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:46.586888 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4b6b4-f9a8-4aca-9fac-22be278faa57 None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:46.590254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4b6b4-f9a8-4aca-9fac-22be278faa57 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:46.594216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 52/114] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:39:46 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 472 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:46.605938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2926c6c-de6d-4bbb-b2bf-6969f4a92448 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:46.606888 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2926c6c-de6d-4bbb-b2bf-6969f4a92448 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:46.607702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2926c6c-de6d-4bbb-b2bf-6969f4a92448 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:46.610769 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 53/115] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:46.612723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2926c6c-de6d-4bbb-b2bf-6969f4a92448 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:46.780723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e69912-6b09-48a8-9836-ec84bb901e4e None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:46.819323 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2926c6c-de6d-4bbb-b2bf-6969f4a92448 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:46.855200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6cdf5-103e-4441-b0be-c63f4cbd4408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:46.857506 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6cdf5-103e-4441-b0be-c63f4cbd4408 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:46.858208 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6cdf5-103e-4441-b0be-c63f4cbd4408 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:46.858919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6cdf5-103e-4441-b0be-c63f4cbd4408 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:46.883402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6cdf5-103e-4441-b0be-c63f4cbd4408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:46.883845 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857859d-445f-4f1c-94b6-17ff580abb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:46.885119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857859d-445f-4f1c-94b6-17ff580abb8c None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:46.885743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857859d-445f-4f1c-94b6-17ff580abb8c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:46.886358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857859d-445f-4f1c-94b6-17ff580abb8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-b3d6cdf5-103e-4441-b0be-c63f4cbd4408 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:46.895898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:46.898487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:46.900373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:46.900373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:46.900373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:46.900373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:46.900373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:46.900373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:46.900373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:46.900373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:46.900373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:46.900373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:46.900373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:39:46.900373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:39:46.904310 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 63/116] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:39:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:46.906098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6cdf5-103e-4441-b0be-c63f4cbd4408 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:46.927447 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:46.956415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:47.034887 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857859d-445f-4f1c-94b6-17ff580abb8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:47.036385 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857859d-445f-4f1c-94b6-17ff580abb8c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:47.126382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6cdf5-103e-4441-b0be-c63f4cbd4408 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:47.169992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 54/117] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:39:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:39:47.171346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b213b0-14e4-468a-a3ff-220be6e4cd19 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:47.176569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b213b0-14e4-468a-a3ff-220be6e4cd19 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:47.176959 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b213b0-14e4-468a-a3ff-220be6e4cd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:47.198748 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-69b213b0-14e4-468a-a3ff-220be6e4cd19 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:47.200159 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-69b213b0-14e4-468a-a3ff-220be6e4cd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:47.200688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1e873-1d4f-416d-81d2-92ebc2119adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:47.201883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1e873-1d4f-416d-81d2-92ebc2119adf None admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:47.202369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1e873-1d4f-416d-81d2-92ebc2119adf None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:47.202868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1e873-1d4f-416d-81d2-92ebc2119adf None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a/users/843d8bc8740f48aaa93c0f79dc9687d0/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:47.206622 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec1e873-1d4f-416d-81d2-92ebc2119adf None admin] RBAC: Authorizing `identity:create_grant(project_id=d0c721e2529541e5966d3c21426e447a, user_id=843d8bc8740f48aaa93c0f79dc9687d0, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:47.210037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec1e873-1d4f-416d-81d2-92ebc2119adf None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:47.281721 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:47.285395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 64/118] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:39:46 2022] PUT /identity/v3/projects/d0c721e2529541e5966d3c21426e447a/users/843d8bc8740f48aaa93c0f79dc9687d0/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:39:47.286673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1e873-1d4f-416d-81d2-92ebc2119adf None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:47.307275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 55/119] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:47.308609 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b213b0-14e4-468a-a3ff-220be6e4cd19 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:47.449273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:47.471091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b213b0-14e4-468a-a3ff-220be6e4cd19 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:47.502403 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1e873-1d4f-416d-81d2-92ebc2119adf None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:47.519898 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bafadd-c93b-4bde-9668-b354f08a077b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:47.521000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bafadd-c93b-4bde-9668-b354f08a077b None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:47.521544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bafadd-c93b-4bde-9668-b354f08a077b None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:47.522040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bafadd-c93b-4bde-9668-b354f08a077b None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:47.523924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bafadd-c93b-4bde-9668-b354f08a077b None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:47.525703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bafadd-c93b-4bde-9668-b354f08a077b None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:47.527934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 56/120] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:39:47 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:47.529442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085245c-7fc9-4c50-9f27-b8b0a57ea6ae None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:47.529945 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085245c-7fc9-4c50-9f27-b8b0a57ea6ae None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:47.530429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085245c-7fc9-4c50-9f27-b8b0a57ea6ae None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:47.532450 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 57/121] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:47.533509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085245c-7fc9-4c50-9f27-b8b0a57ea6ae None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:47.555555 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a34d7-8abd-4e1d-9c6f-ae8e642c1191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:47.556284 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a34d7-8abd-4e1d-9c6f-ae8e642c1191 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:47.556566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a34d7-8abd-4e1d-9c6f-ae8e642c1191 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:47.556863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a34d7-8abd-4e1d-9c6f-ae8e642c1191 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:47.557610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a34d7-8abd-4e1d-9c6f-ae8e642c1191 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:47.557928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a34d7-8abd-4e1d-9c6f-ae8e642c1191 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:47.559344 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a34d7-8abd-4e1d-9c6f-ae8e642c1191 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:47.570207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 65/122] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:39:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:47.571199 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a34d7-8abd-4e1d-9c6f-ae8e642c1191 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:47.627415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:47.686994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085245c-7fc9-4c50-9f27-b8b0a57ea6ae None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:47.727034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a34d7-8abd-4e1d-9c6f-ae8e642c1191 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:47.733757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71575fd-0692-46ad-9138-3df1418168ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:47.734433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71575fd-0692-46ad-9138-3df1418168ea None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:47.734723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71575fd-0692-46ad-9138-3df1418168ea None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:47.735092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71575fd-0692-46ad-9138-3df1418168ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:47.761870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71575fd-0692-46ad-9138-3df1418168ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:47.769832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-a71575fd-0692-46ad-9138-3df1418168ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:47.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:47.770835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:47.771701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:47.771701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:47.771701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:47.771701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:47.771701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:47.771701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:47.771701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:47.771701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:47.771701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:47.771701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:47.771701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:39:47.771701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:39:47.771701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 58/123] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:47.774876 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94836d8f-5de7-4a28-8e1a-fec0ae50f09b None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:47.775172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94836d8f-5de7-4a28-8e1a-fec0ae50f09b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:47.775482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94836d8f-5de7-4a28-8e1a-fec0ae50f09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:47.784578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6bca3-418d-4dbc-b867-fde7410f1bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:47.788267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6bca3-418d-4dbc-b867-fde7410f1bef None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:47.788267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6bca3-418d-4dbc-b867-fde7410f1bef None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:47.788267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6bca3-418d-4dbc-b867-fde7410f1bef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:47.788267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d6bca3-418d-4dbc-b867-fde7410f1bef None admin] RBAC: Adding query filter params (role.id=cf7c52610e954f74a30feba325390387, scope.project.id=d0c721e2529541e5966d3c21426e447a, user.id=84fbc773412c4d25a3aedf5914c636fc) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:47.788990 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d6bca3-418d-4dbc-b867-fde7410f1bef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:47.790867 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d6bca3-418d-4dbc-b867-fde7410f1bef None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:47.797494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 66/124] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:39:47 2022] GET /identity/v3/role_assignments?user.id=84fbc773412c4d25a3aedf5914c636fc&scope.project.id=d0c721e2529541e5966d3c21426e447a&role.id=cf7c52610e954f74a30feba325390387 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:47.803554 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-94836d8f-5de7-4a28-8e1a-fec0ae50f09b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:47.803554 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-94836d8f-5de7-4a28-8e1a-fec0ae50f09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:47.804656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6bca3-418d-4dbc-b867-fde7410f1bef None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:47.949901 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 59/125] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:47.952808 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94836d8f-5de7-4a28-8e1a-fec0ae50f09b None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:48.073663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6bca3-418d-4dbc-b867-fde7410f1bef None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:48.088166 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:48.196766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981b4b4-72a4-4d76-a506-a069cd991395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:48.197885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981b4b4-72a4-4d76-a506-a069cd991395 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:48.198365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981b4b4-72a4-4d76-a506-a069cd991395 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:48.198860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981b4b4-72a4-4d76-a506-a069cd991395 None admin] PATH_INFO: `/v3/users/843d8bc8740f48aaa93c0f79dc9687d0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:48.202673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b981b4b4-72a4-4d76-a506-a069cd991395 None admin] RBAC: Authorizing `identity:get_user(user_id=843d8bc8740f48aaa93c0f79dc9687d0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:48.205682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b981b4b4-72a4-4d76-a506-a069cd991395 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:48.209801 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 67/126] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:47 2022] GET /identity/v3/users/843d8bc8740f48aaa93c0f79dc9687d0 => generated 275 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:48.210968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981b4b4-72a4-4d76-a506-a069cd991395 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:48.221046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94836d8f-5de7-4a28-8e1a-fec0ae50f09b None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:48.337367 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df038426-84d0-4980-aae6-3e8147449021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:48.338439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df038426-84d0-4980-aae6-3e8147449021 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:48.338909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df038426-84d0-4980-aae6-3e8147449021 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:48.339395 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df038426-84d0-4980-aae6-3e8147449021 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:48.340829 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df038426-84d0-4980-aae6-3e8147449021 None admin] RBAC: Adding query filter params (name=member) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:48.341357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df038426-84d0-4980-aae6-3e8147449021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:48.344009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df038426-84d0-4980-aae6-3e8147449021 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:48.382372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 60/127] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:39:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:48.383944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df038426-84d0-4980-aae6-3e8147449021 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:48.457863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:48.606619 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981b4b4-72a4-4d76-a506-a069cd991395 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:48.685686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df038426-84d0-4980-aae6-3e8147449021 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:48.689104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6df5f3-eeb4-4717-92cf-d7fb11aa1992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:48.690418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6df5f3-eeb4-4717-92cf-d7fb11aa1992 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:48.690901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6df5f3-eeb4-4717-92cf-d7fb11aa1992 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:48.691406 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6df5f3-eeb4-4717-92cf-d7fb11aa1992 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:48.731656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6df5f3-eeb4-4717-92cf-d7fb11aa1992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:48.731656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-6a6df5f3-eeb4-4717-92cf-d7fb11aa1992 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:39:48.731656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:48.731656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:48.731656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:48.731656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:48.731656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:48.731656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:48.731656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:48.731656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:48.731656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:48.731656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:48.731656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:48.731656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:48.733608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:39:48.735936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 68/128] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:39:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 513 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:48.738796 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6df5f3-eeb4-4717-92cf-d7fb11aa1992 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:48.739610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:48.834592 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b6947-a6af-46d2-867d-4834a40a0d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:48.834922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b6947-a6af-46d2-867d-4834a40a0d0a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:48.835212 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b6947-a6af-46d2-867d-4834a40a0d0a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:48.835494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b6947-a6af-46d2-867d-4834a40a0d0a None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:48.844490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b6947-a6af-46d2-867d-4834a40a0d0a None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:48.846188 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b6947-a6af-46d2-867d-4834a40a0d0a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:48.848149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 61/129] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:39:48 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:48.850023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b6947-a6af-46d2-867d-4834a40a0d0a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:48.967009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6df5f3-eeb4-4717-92cf-d7fb11aa1992 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:49.009276 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:49.033054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90af80-f51a-404d-b7e1-5d6570fbfd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:49.033692 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90af80-f51a-404d-b7e1-5d6570fbfd6f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:49.033976 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90af80-f51a-404d-b7e1-5d6570fbfd6f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:49.034257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90af80-f51a-404d-b7e1-5d6570fbfd6f None admin] PATH_INFO: `/v3/users/843d8bc8740f48aaa93c0f79dc9687d0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:49.035684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90af80-f51a-404d-b7e1-5d6570fbfd6f None admin] RBAC: Authorizing `identity:get_user(user_id=843d8bc8740f48aaa93c0f79dc9687d0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:49.038551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90af80-f51a-404d-b7e1-5d6570fbfd6f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:49.040653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 69/130] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:48 2022] GET /identity/v3/users/843d8bc8740f48aaa93c0f79dc9687d0 => generated 275 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:49.045110 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90af80-f51a-404d-b7e1-5d6570fbfd6f None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:49.090278 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b6947-a6af-46d2-867d-4834a40a0d0a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:49.152430 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8540f-90a3-45c4-a8c9-92f6f516a3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:49.153277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8540f-90a3-45c4-a8c9-92f6f516a3a7 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:49.153668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8540f-90a3-45c4-a8c9-92f6f516a3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:49.153951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8540f-90a3-45c4-a8c9-92f6f516a3a7 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:49.154929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8540f-90a3-45c4-a8c9-92f6f516a3a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:49.155292 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8540f-90a3-45c4-a8c9-92f6f516a3a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:49.157339 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8540f-90a3-45c4-a8c9-92f6f516a3a7 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:49.170548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 62/131] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:49.178289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ed6cc-016e-4355-b50c-569f0c5cbffd None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:49.178805 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ed6cc-016e-4355-b50c-569f0c5cbffd None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:49.179269 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ed6cc-016e-4355-b50c-569f0c5cbffd None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:49.181212 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 63/132] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:49 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:49.187496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ed6cc-016e-4355-b50c-569f0c5cbffd None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:49.233461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90af80-f51a-404d-b7e1-5d6570fbfd6f None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:49.257014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:49.296544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5062586-3d7a-4e13-9196-ef71bb08604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:49.298795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5062586-3d7a-4e13-9196-ef71bb08604a None admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:49.299292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5062586-3d7a-4e13-9196-ef71bb08604a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:49.300055 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5062586-3d7a-4e13-9196-ef71bb08604a None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a/users/843d8bc8740f48aaa93c0f79dc9687d0/roles/eff5cca513be4ed5a9b001c0559fb476` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:49.303579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5062586-3d7a-4e13-9196-ef71bb08604a None admin] RBAC: Authorizing `identity:create_grant(project_id=d0c721e2529541e5966d3c21426e447a, user_id=843d8bc8740f48aaa93c0f79dc9687d0, role_id=eff5cca513be4ed5a9b001c0559fb476)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:49.305673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5062586-3d7a-4e13-9196-ef71bb08604a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:49.345773 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ed6cc-016e-4355-b50c-569f0c5cbffd None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:49.391222 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea2666-7459-4bcc-ade9-874066eb14a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:49.392031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea2666-7459-4bcc-ade9-874066eb14a4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:49.392316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea2666-7459-4bcc-ade9-874066eb14a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:49.392601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea2666-7459-4bcc-ade9-874066eb14a4 None admin] PATH_INFO: `/v3/projects/66ad66f40fde439f881153a21f8c2b44` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:49.393940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea2666-7459-4bcc-ade9-874066eb14a4 None admin] RBAC: Authorizing `identity:get_project(project_id=66ad66f40fde439f881153a21f8c2b44)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:49.395417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea2666-7459-4bcc-ade9-874066eb14a4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:49.401731 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 64/133] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:39:49 2022] GET /identity/v3/projects/66ad66f40fde439f881153a21f8c2b44 => generated 312 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:49.401731 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588755c5-ee2b-49ba-a0f6-c9206fb40664 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:49.401731 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588755c5-ee2b-49ba-a0f6-c9206fb40664 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:49.401731 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588755c5-ee2b-49ba-a0f6-c9206fb40664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:49.423019 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 70/134] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:39:49 2022] PUT /identity/v3/projects/d0c721e2529541e5966d3c21426e447a/users/843d8bc8740f48aaa93c0f79dc9687d0/roles/eff5cca513be4ed5a9b001c0559fb476 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:39:49.428029 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5062586-3d7a-4e13-9196-ef71bb08604a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:49.442180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-588755c5-ee2b-49ba-a0f6-c9206fb40664 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:49.454177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-588755c5-ee2b-49ba-a0f6-c9206fb40664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:49.599137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:49.663663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 65/135] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:49.667374 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588755c5-ee2b-49ba-a0f6-c9206fb40664 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:49.778719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5062586-3d7a-4e13-9196-ef71bb08604a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:49.794117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:49.859578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45397c48-18eb-402f-835b-9ff2fdd66064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:49.861076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45397c48-18eb-402f-835b-9ff2fdd66064 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:49.861808 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45397c48-18eb-402f-835b-9ff2fdd66064 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:49.862541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45397c48-18eb-402f-835b-9ff2fdd66064 None admin] PATH_INFO: `/v3/users/31dac45eb8764daf9a148a750fdc78a0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:49.865789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45397c48-18eb-402f-835b-9ff2fdd66064 None admin] RBAC: Authorizing `identity:get_user(user_id=31dac45eb8764daf9a148a750fdc78a0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:49.868908 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45397c48-18eb-402f-835b-9ff2fdd66064 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:49.875806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588755c5-ee2b-49ba-a0f6-c9206fb40664 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:49.876303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 71/136] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:49 2022] GET /identity/v3/users/31dac45eb8764daf9a148a750fdc78a0 => generated 247 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:49.878168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45397c48-18eb-402f-835b-9ff2fdd66064 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:49.920722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130352c5-574f-420c-82b2-f17313ae24c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:49.921936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130352c5-574f-420c-82b2-f17313ae24c2 None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:49.921936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130352c5-574f-420c-82b2-f17313ae24c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:49.922237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130352c5-574f-420c-82b2-f17313ae24c2 None admin] PATH_INFO: `/v3/projects/66ad66f40fde439f881153a21f8c2b44/users/843d8bc8740f48aaa93c0f79dc9687d0/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:49.924956 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130352c5-574f-420c-82b2-f17313ae24c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=66ad66f40fde439f881153a21f8c2b44, user_id=843d8bc8740f48aaa93c0f79dc9687d0, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:49.926693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130352c5-574f-420c-82b2-f17313ae24c2 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:50.025440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 66/137] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:39:49 2022] PUT /identity/v3/projects/66ad66f40fde439f881153a21f8c2b44/users/843d8bc8740f48aaa93c0f79dc9687d0/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:39:50.032126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130352c5-574f-420c-82b2-f17313ae24c2 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:50.312118 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45397c48-18eb-402f-835b-9ff2fdd66064 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:50.339984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:50.350950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130352c5-574f-420c-82b2-f17313ae24c2 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:50.389740 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:50.451442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd867df5-6d74-464e-bf45-03164f4ded7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:50.452997 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd867df5-6d74-464e-bf45-03164f4ded7e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:50.454464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd867df5-6d74-464e-bf45-03164f4ded7e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:50.455286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa0794-1dab-42ab-a919-3d5266ce22ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:50.455781 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd867df5-6d74-464e-bf45-03164f4ded7e None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:50.457373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa0794-1dab-42ab-a919-3d5266ce22ec None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:50.458258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd867df5-6d74-464e-bf45-03164f4ded7e None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:50.459382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa0794-1dab-42ab-a919-3d5266ce22ec None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:50.465456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd867df5-6d74-464e-bf45-03164f4ded7e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:50.466083 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa0794-1dab-42ab-a919-3d5266ce22ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:50.467722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa0794-1dab-42ab-a919-3d5266ce22ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:50.469918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 67/138] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:39:50 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:50.472007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa0794-1dab-42ab-a919-3d5266ce22ec None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:50.474047 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 72/139] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:39:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 597 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:50.483277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd867df5-6d74-464e-bf45-03164f4ded7e None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:50.485857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa0794-1dab-42ab-a919-3d5266ce22ec None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:50.866621 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd867df5-6d74-464e-bf45-03164f4ded7e None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:50.873963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa0794-1dab-42ab-a919-3d5266ce22ec None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:50.950894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79298290-095e-41d6-a853-e2ce6b24929f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:50.952081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79298290-095e-41d6-a853-e2ce6b24929f None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:50.952495 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79298290-095e-41d6-a853-e2ce6b24929f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:50.952892 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79298290-095e-41d6-a853-e2ce6b24929f None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a/users/31dac45eb8764daf9a148a750fdc78a0/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:50.958440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79298290-095e-41d6-a853-e2ce6b24929f None admin] RBAC: Authorizing `identity:create_grant(project_id=d0c721e2529541e5966d3c21426e447a, user_id=31dac45eb8764daf9a148a750fdc78a0, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:50.961700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79298290-095e-41d6-a853-e2ce6b24929f None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:50.994331 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7fe13-abd8-4695-a8e9-2b8e123ab538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:51.000016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7fe13-abd8-4695-a8e9-2b8e123ab538 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:51.000016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7fe13-abd8-4695-a8e9-2b8e123ab538 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:51.000016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7fe13-abd8-4695-a8e9-2b8e123ab538 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:51.000016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7fe13-abd8-4695-a8e9-2b8e123ab538 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:51.000680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7fe13-abd8-4695-a8e9-2b8e123ab538 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:51.066543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 68/140] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:39:50 2022] PUT /identity/v3/projects/d0c721e2529541e5966d3c21426e447a/users/31dac45eb8764daf9a148a750fdc78a0/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:39:51.070313 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae04137-3049-4914-824f-7770f1436a51 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:51.070815 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae04137-3049-4914-824f-7770f1436a51 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:51.071359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae04137-3049-4914-824f-7770f1436a51 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:51.073745 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 69/141] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:51.086703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be55747-f0f9-4b2a-a4e0-a77658ca925d None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:51.087229 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be55747-f0f9-4b2a-a4e0-a77658ca925d None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:51.087836 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be55747-f0f9-4b2a-a4e0-a77658ca925d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:51.108786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 73/142] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:39:50 2022] POST /identity/v3/users => generated 283 bytes in 623 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:39:51.134601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-1be55747-f0f9-4b2a-a4e0-a77658ca925d None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:51.136365 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1be55747-f0f9-4b2a-a4e0-a77658ca925d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:51.197315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:51.324005 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 70/143] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:51.351904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fc1f6-b7db-45aa-a15d-9f751a61d2a7 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:51.352458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fc1f6-b7db-45aa-a15d-9f751a61d2a7 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:51.352959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fc1f6-b7db-45aa-a15d-9f751a61d2a7 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:51.355819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 74/144] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:51.366729 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca7de7-4015-49ab-89e3-e218a848c2bf None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:51.367306 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca7de7-4015-49ab-89e3-e218a848c2bf None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:51.367810 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca7de7-4015-49ab-89e3-e218a848c2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:51.372650 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fc1f6-b7db-45aa-a15d-9f751a61d2a7 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:51.429659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-d0ca7de7-4015-49ab-89e3-e218a848c2bf None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:51.433020 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d0ca7de7-4015-49ab-89e3-e218a848c2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:51.497116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:51.585720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 71/145] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:51.609460 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fc1f6-b7db-45aa-a15d-9f751a61d2a7 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:51.667502 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca7de7-4015-49ab-89e3-e218a848c2bf None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:51.696244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44e5cb-c9e3-4535-a054-a1ff6f63a915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:51.696915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44e5cb-c9e3-4535-a054-a1ff6f63a915 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:51.697203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44e5cb-c9e3-4535-a054-a1ff6f63a915 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:51.697469 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44e5cb-c9e3-4535-a054-a1ff6f63a915 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:51.740641 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44e5cb-c9e3-4535-a054-a1ff6f63a915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-ad44e5cb-c9e3-4535-a054-a1ff6f63a915 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:51.751418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:51.753237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:51.755401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:51.755401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:51.755401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:51.755401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:51.755401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:51.755401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:51.755401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:51.755401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:51.755401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:51.755401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:51.755401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:39:51.755401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:39:51.755401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 75/146] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:51.764537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44e5cb-c9e3-4535-a054-a1ff6f63a915 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:51.889137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:51.921249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca7de7-4015-49ab-89e3-e218a848c2bf None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:52.010636 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50236e-aedc-4649-8d4f-f6a0aabff54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:52.011296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50236e-aedc-4649-8d4f-f6a0aabff54e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:52.011715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50236e-aedc-4649-8d4f-f6a0aabff54e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:52.012039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50236e-aedc-4649-8d4f-f6a0aabff54e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:52.038570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50236e-aedc-4649-8d4f-f6a0aabff54e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:52.039941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:52.043239 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44e5cb-c9e3-4535-a054-a1ff6f63a915 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-cf50236e-aedc-4649-8d4f-f6a0aabff54e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:52.045183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:52.046790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:52.057180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:52.057180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:52.057180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:52.057180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:52.057180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:52.057180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:52.057180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:52.057180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:52.057180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:52.057180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:52.057180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:39:52.057180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:39:52.057180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 72/147] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:39:52.069358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50236e-aedc-4649-8d4f-f6a0aabff54e None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:52.103304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf94943-f248-469d-9f8d-1a76b28c99ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:52.103981 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf94943-f248-469d-9f8d-1a76b28c99ef None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:52.104360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf94943-f248-469d-9f8d-1a76b28c99ef None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:52.104827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf94943-f248-469d-9f8d-1a76b28c99ef None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:52.105925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf94943-f248-469d-9f8d-1a76b28c99ef None admin] RBAC: Adding query filter params (name=member) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:52.106245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf94943-f248-469d-9f8d-1a76b28c99ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:52.107962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf94943-f248-469d-9f8d-1a76b28c99ef None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:52.119978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 76/148] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:39:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:52.126811 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf94943-f248-469d-9f8d-1a76b28c99ef None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:52.380804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf94943-f248-469d-9f8d-1a76b28c99ef None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:52.381766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50236e-aedc-4649-8d4f-f6a0aabff54e None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:52.496769 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd22b1-b63e-41eb-9cdf-b65c6f9cbb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:52.497871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd22b1-b63e-41eb-9cdf-b65c6f9cbb3d None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:52.498427 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd22b1-b63e-41eb-9cdf-b65c6f9cbb3d None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:52.498950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd22b1-b63e-41eb-9cdf-b65c6f9cbb3d None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:52.500513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd22b1-b63e-41eb-9cdf-b65c6f9cbb3d None admin] RBAC: Adding query filter params (name=reader) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:52.501098 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd22b1-b63e-41eb-9cdf-b65c6f9cbb3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:52.504692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd22b1-b63e-41eb-9cdf-b65c6f9cbb3d None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:52.508494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983a583-d6d9-40ef-bb09-a31496c2eb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:52.509218 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983a583-d6d9-40ef-bb09-a31496c2eb82 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:52.509619 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983a583-d6d9-40ef-bb09-a31496c2eb82 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:52.509901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983a583-d6d9-40ef-bb09-a31496c2eb82 None admin] PATH_INFO: `/v3/users/843d8bc8740f48aaa93c0f79dc9687d0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:52.511436 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983a583-d6d9-40ef-bb09-a31496c2eb82 None admin] RBAC: Authorizing `identity:get_user(user_id=843d8bc8740f48aaa93c0f79dc9687d0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:52.513014 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983a583-d6d9-40ef-bb09-a31496c2eb82 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:52.515210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 77/149] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:52 2022] GET /identity/v3/users/843d8bc8740f48aaa93c0f79dc9687d0 => generated 275 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:52.523434 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983a583-d6d9-40ef-bb09-a31496c2eb82 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:52.526445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 73/150] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:39:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:52.533544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd22b1-b63e-41eb-9cdf-b65c6f9cbb3d None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:52.578567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:52.631764 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:52.840115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983a583-d6d9-40ef-bb09-a31496c2eb82 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:52.852539 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd22b1-b63e-41eb-9cdf-b65c6f9cbb3d None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:52.932923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebd8de-585e-4d09-a8a2-fceb911a4e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:52.933834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebd8de-585e-4d09-a8a2-fceb911a4e6e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:52.934297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebd8de-585e-4d09-a8a2-fceb911a4e6e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:52.934761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebd8de-585e-4d09-a8a2-fceb911a4e6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:52.976034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f49ed-3b8d-4d91-b5dc-7402a2a480ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:52.977320 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f49ed-3b8d-4d91-b5dc-7402a2a480ce None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:52.977850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f49ed-3b8d-4d91-b5dc-7402a2a480ce None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:52.978395 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f49ed-3b8d-4d91-b5dc-7402a2a480ce None admin] PATH_INFO: `/v3/users/84fbc773412c4d25a3aedf5914c636fc` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:52.980774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f49ed-3b8d-4d91-b5dc-7402a2a480ce None admin] RBAC: Authorizing `identity:get_user(user_id=84fbc773412c4d25a3aedf5914c636fc)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:52.983894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f49ed-3b8d-4d91-b5dc-7402a2a480ce None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:52.987064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 74/151] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:52 2022] GET /identity/v3/users/84fbc773412c4d25a3aedf5914c636fc => generated 289 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:52.996594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f49ed-3b8d-4d91-b5dc-7402a2a480ce None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:53.120357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebd8de-585e-4d09-a8a2-fceb911a4e6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:53.121993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebd8de-585e-4d09-a8a2-fceb911a4e6e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:53.167649 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:53.196208 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:53.213519 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f49ed-3b8d-4d91-b5dc-7402a2a480ce None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:53.283027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7bc80-e579-4013-b550-6a717d81a998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:53.284168 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7bc80-e579-4013-b550-6a717d81a998 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:53.284693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7bc80-e579-4013-b550-6a717d81a998 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:53.285246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7bc80-e579-4013-b550-6a717d81a998 None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:53.288779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7bc80-e579-4013-b550-6a717d81a998 None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:53.291675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7bc80-e579-4013-b550-6a717d81a998 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:53.295373 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 75/152] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:39:52 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:53.304801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7bc80-e579-4013-b550-6a717d81a998 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:53.366458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 78/153] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:39:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 843 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:39:53.372278 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebd8de-585e-4d09-a8a2-fceb911a4e6e None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:53.516174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7bc80-e579-4013-b550-6a717d81a998 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:53.624739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebd8de-585e-4d09-a8a2-fceb911a4e6e None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:53.628886 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a70b75-d514-45dd-8ad7-1c6b2a218905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:53.629594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a70b75-d514-45dd-8ad7-1c6b2a218905 None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:53.629878 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a70b75-d514-45dd-8ad7-1c6b2a218905 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:53.630158 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a70b75-d514-45dd-8ad7-1c6b2a218905 None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a/users/84fbc773412c4d25a3aedf5914c636fc/roles/cf7c52610e954f74a30feba325390387` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:53.634527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a70b75-d514-45dd-8ad7-1c6b2a218905 None admin] RBAC: Authorizing `identity:create_grant(project_id=d0c721e2529541e5966d3c21426e447a, user_id=84fbc773412c4d25a3aedf5914c636fc, role_id=cf7c52610e954f74a30feba325390387)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:53.636122 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a70b75-d514-45dd-8ad7-1c6b2a218905 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:53.688949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:53.696733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d9629-8681-4fe4-b6f3-c27e726b88ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:53.697423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d9629-8681-4fe4-b6f3-c27e726b88ec None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:53.697706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d9629-8681-4fe4-b6f3-c27e726b88ec None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:53.698028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d9629-8681-4fe4-b6f3-c27e726b88ec None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:53.704682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d9629-8681-4fe4-b6f3-c27e726b88ec None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:53.706241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d9629-8681-4fe4-b6f3-c27e726b88ec None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:53.708518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 79/154] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:39:53 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:53.715318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d9629-8681-4fe4-b6f3-c27e726b88ec None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:53.771488 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 76/155] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:39:53 2022] PUT /identity/v3/projects/d0c721e2529541e5966d3c21426e447a/users/84fbc773412c4d25a3aedf5914c636fc/roles/cf7c52610e954f74a30feba325390387 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:39:53.959189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d9629-8681-4fe4-b6f3-c27e726b88ec None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:54.036470 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b71b21-88a9-469f-a0b9-bddd23435e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:54.039580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b71b21-88a9-469f-a0b9-bddd23435e35 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:54.039580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b71b21-88a9-469f-a0b9-bddd23435e35 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:54.039580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b71b21-88a9-469f-a0b9-bddd23435e35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:54.040865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b71b21-88a9-469f-a0b9-bddd23435e35 None admin] RBAC: Adding query filter params (role.id=6a924f1d619643b3be92bab6fddb3eb2, scope.project.id=d0c721e2529541e5966d3c21426e447a, user.id=843d8bc8740f48aaa93c0f79dc9687d0) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:54.041407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b71b21-88a9-469f-a0b9-bddd23435e35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:54.044212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b71b21-88a9-469f-a0b9-bddd23435e35 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:54.056884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 80/156] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:39:53 2022] GET /identity/v3/role_assignments?user.id=843d8bc8740f48aaa93c0f79dc9687d0&scope.project.id=d0c721e2529541e5966d3c21426e447a&role.id=6a924f1d619643b3be92bab6fddb3eb2 => generated 608 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:54.251760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18ce7f-cdfa-4524-8c7e-c00a61a87ddb None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:54.252063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18ce7f-cdfa-4524-8c7e-c00a61a87ddb None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:54.252326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18ce7f-cdfa-4524-8c7e-c00a61a87ddb None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:54.254543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 77/157] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:54.266234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0d2f2-9721-400d-b927-0137419ef7e6 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:54.266234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0d2f2-9721-400d-b927-0137419ef7e6 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:54.266234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0d2f2-9721-400d-b927-0137419ef7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:54.297440 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-42a0d2f2-9721-400d-b927-0137419ef7e6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:54.299342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-42a0d2f2-9721-400d-b927-0137419ef7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:54.420792 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:54.447534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 81/158] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:39:54.506893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18ce7f-cdfa-4524-8c7e-c00a61a87ddb None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:54.524327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:54.670884 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18ce7f-cdfa-4524-8c7e-c00a61a87ddb None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:54.723741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafea92a-2346-4ffd-9aa8-6da1045ccb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:54.725074 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafea92a-2346-4ffd-9aa8-6da1045ccb78 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:54.725363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafea92a-2346-4ffd-9aa8-6da1045ccb78 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:54.725828 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafea92a-2346-4ffd-9aa8-6da1045ccb78 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:54.740889 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafea92a-2346-4ffd-9aa8-6da1045ccb78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-cafea92a-2346-4ffd-9aa8-6da1045ccb78 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:54.747391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:54.749351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:54.751084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:54.751084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:54.751084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:54.751084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:54.751084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:54.751084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:54.751084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:54.751084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:54.751084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:54.751084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:54.751084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:39:54.751084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:39:54.751084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 78/159] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:39:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:54.762037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0d2f2-9721-400d-b927-0137419ef7e6 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:54.775354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf5e00-caf7-4e36-8d5c-4894c01553a2 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:54.775681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf5e00-caf7-4e36-8d5c-4894c01553a2 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:54.775942 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf5e00-caf7-4e36-8d5c-4894c01553a2 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:54.777780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 79/160] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:39:54.790426 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a280ef-6e49-4363-bdf2-647f85293e23 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:54.790697 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a280ef-6e49-4363-bdf2-647f85293e23 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:54.791017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a280ef-6e49-4363-bdf2-647f85293e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:54.824873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-b1a280ef-6e49-4363-bdf2-647f85293e23 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:54.826134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b1a280ef-6e49-4363-bdf2-647f85293e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:54.964455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:54.978233 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 80/161] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:39:55.048624 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0d2f2-9721-400d-b927-0137419ef7e6 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:55.056921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a280ef-6e49-4363-bdf2-647f85293e23 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:55.125892 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e12b51-d417-4580-b071-24c245926224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:55.127303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e12b51-d417-4580-b071-24c245926224 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:55.127808 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e12b51-d417-4580-b071-24c245926224 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:55.128280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e12b51-d417-4580-b071-24c245926224 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:55.129512 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e12b51-d417-4580-b071-24c245926224 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:55.130033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e12b51-d417-4580-b071-24c245926224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:55.134816 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e12b51-d417-4580-b071-24c245926224 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:55.152703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 82/162] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:39:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:55.160381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e12b51-d417-4580-b071-24c245926224 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:55.314432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:55.364104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a280ef-6e49-4363-bdf2-647f85293e23 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:55.463388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e12b51-d417-4580-b071-24c245926224 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:55.464543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f943e89-a810-40b1-8478-546316a7cf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:55.465469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f943e89-a810-40b1-8478-546316a7cf4f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:55.465919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f943e89-a810-40b1-8478-546316a7cf4f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:55.466390 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f943e89-a810-40b1-8478-546316a7cf4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:55.500664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f943e89-a810-40b1-8478-546316a7cf4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-7f943e89-a810-40b1-8478-546316a7cf4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:55.512511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:55.513549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:55.514432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:55.514432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:55.514432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:55.514432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:55.514432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:55.514432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:55.514432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:55.514432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:55.514432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:55.514432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:55.514432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:39:55.514432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:39:55.516685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 81/163] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:39:55.529751 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f943e89-a810-40b1-8478-546316a7cf4f None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:55.565083 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19662f7-151b-49f9-a142-b4505efbac47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:55.566134 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19662f7-151b-49f9-a142-b4505efbac47 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:55.566616 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19662f7-151b-49f9-a142-b4505efbac47 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:55.567103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19662f7-151b-49f9-a142-b4505efbac47 None admin] PATH_INFO: `/v3/users/843d8bc8740f48aaa93c0f79dc9687d0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:55.570054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19662f7-151b-49f9-a142-b4505efbac47 None admin] RBAC: Authorizing `identity:get_user(user_id=843d8bc8740f48aaa93c0f79dc9687d0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:55.572918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19662f7-151b-49f9-a142-b4505efbac47 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:55.576338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 83/164] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:55 2022] GET /identity/v3/users/843d8bc8740f48aaa93c0f79dc9687d0 => generated 275 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:55.585489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19662f7-151b-49f9-a142-b4505efbac47 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:55.697950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:55.785782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19662f7-151b-49f9-a142-b4505efbac47 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:55.823913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f943e89-a810-40b1-8478-546316a7cf4f None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:55.831007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:55.852031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca564882-c635-4b15-a094-b976774bc794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:55.853023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca564882-c635-4b15-a094-b976774bc794 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:55.853464 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca564882-c635-4b15-a094-b976774bc794 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:55.853902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca564882-c635-4b15-a094-b976774bc794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:55.922136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e876e-44f0-4796-8cba-01f030998712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:55.923254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e876e-44f0-4796-8cba-01f030998712 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:55.923739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e876e-44f0-4796-8cba-01f030998712 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:55.924180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e876e-44f0-4796-8cba-01f030998712 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:55.925475 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e876e-44f0-4796-8cba-01f030998712 None admin] RBAC: Adding query filter params (name=member) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:55.925983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e876e-44f0-4796-8cba-01f030998712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:55.928636 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e876e-44f0-4796-8cba-01f030998712 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:55.956517 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 82/165] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:39:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:55.964929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e876e-44f0-4796-8cba-01f030998712 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:56.007914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca564882-c635-4b15-a094-b976774bc794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:56.010345 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca564882-c635-4b15-a094-b976774bc794 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:56.202417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e876e-44f0-4796-8cba-01f030998712 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:56.240411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:56.276917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31702c99-b861-4af3-b3b5-7d1d036db512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:56.278711 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31702c99-b861-4af3-b3b5-7d1d036db512 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:56.279528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31702c99-b861-4af3-b3b5-7d1d036db512 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:56.280744 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31702c99-b861-4af3-b3b5-7d1d036db512 None admin] PATH_INFO: `/v3/users/843d8bc8740f48aaa93c0f79dc9687d0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:56.284197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31702c99-b861-4af3-b3b5-7d1d036db512 None admin] RBAC: Authorizing `identity:get_user(user_id=843d8bc8740f48aaa93c0f79dc9687d0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:56.288483 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31702c99-b861-4af3-b3b5-7d1d036db512 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:56.292291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 83/166] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:55 2022] GET /identity/v3/users/843d8bc8740f48aaa93c0f79dc9687d0 => generated 275 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:56.301774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31702c99-b861-4af3-b3b5-7d1d036db512 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:56.305672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 84/167] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:39:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 14:39:56.313665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca564882-c635-4b15-a094-b976774bc794 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:56.355340 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:56.464653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca564882-c635-4b15-a094-b976774bc794 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:56.522080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890c0d9-2941-4d94-bb2d-72b3eaeefa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:56.522926 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890c0d9-2941-4d94-bb2d-72b3eaeefa89 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:56.523302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890c0d9-2941-4d94-bb2d-72b3eaeefa89 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:56.523715 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890c0d9-2941-4d94-bb2d-72b3eaeefa89 None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:56.528985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890c0d9-2941-4d94-bb2d-72b3eaeefa89 None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:56.530640 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890c0d9-2941-4d94-bb2d-72b3eaeefa89 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:56.533342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 85/168] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:39:56 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:56.535307 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31702c99-b861-4af3-b3b5-7d1d036db512 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:56.541724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890c0d9-2941-4d94-bb2d-72b3eaeefa89 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:56.611578 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95080aa3-d6c3-49fe-8de3-b184f088e738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:56.612943 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95080aa3-d6c3-49fe-8de3-b184f088e738 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:56.613631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95080aa3-d6c3-49fe-8de3-b184f088e738 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:56.614406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95080aa3-d6c3-49fe-8de3-b184f088e738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:56.726237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:56.732259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:56.749006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890c0d9-2941-4d94-bb2d-72b3eaeefa89 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:56.751746 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95080aa3-d6c3-49fe-8de3-b184f088e738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:56.754376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95080aa3-d6c3-49fe-8de3-b184f088e738 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:56.814321 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d88747-c8e9-4052-a334-00b2e520834f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:56.815089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d88747-c8e9-4052-a334-00b2e520834f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:56.815454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d88747-c8e9-4052-a334-00b2e520834f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:56.815939 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d88747-c8e9-4052-a334-00b2e520834f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:56.817034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d88747-c8e9-4052-a334-00b2e520834f None admin] RBAC: Adding query filter params (role.id=eff5cca513be4ed5a9b001c0559fb476, scope.project.id=d0c721e2529541e5966d3c21426e447a, user.id=843d8bc8740f48aaa93c0f79dc9687d0) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:56.818280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d88747-c8e9-4052-a334-00b2e520834f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:56.820924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d88747-c8e9-4052-a334-00b2e520834f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:56.832459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 86/169] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:39:56 2022] GET /identity/v3/role_assignments?user.id=843d8bc8740f48aaa93c0f79dc9687d0&scope.project.id=d0c721e2529541e5966d3c21426e447a&role.id=eff5cca513be4ed5a9b001c0559fb476 => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:56.834695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2bedf-0800-48f7-bffa-463ba41911aa None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:56.835026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2bedf-0800-48f7-bffa-463ba41911aa None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:56.835299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2bedf-0800-48f7-bffa-463ba41911aa None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:56.836827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 87/170] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:56.850411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef46b4-ddcf-4a43-ad97-a93d69ce47de None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:56.850812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef46b4-ddcf-4a43-ad97-a93d69ce47de None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:56.850979 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef46b4-ddcf-4a43-ad97-a93d69ce47de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:56.908955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b8ef46b4-ddcf-4a43-ad97-a93d69ce47de None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:56.910293 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b8ef46b4-ddcf-4a43-ad97-a93d69ce47de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:56.981544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 84/171] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:39:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 14:39:56.990562 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95080aa3-d6c3-49fe-8de3-b184f088e738 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:57.082210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 88/172] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 26 14:39:57.152372 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef46b4-ddcf-4a43-ad97-a93d69ce47de None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:57.214355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:57.246469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95080aa3-d6c3-49fe-8de3-b184f088e738 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:57.304869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:57.327291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef46b4-ddcf-4a43-ad97-a93d69ce47de None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:57.343385 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc4281-a82d-46e3-bb1b-da18935874c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:57.344123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc4281-a82d-46e3-bb1b-da18935874c2 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:57.344407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc4281-a82d-46e3-bb1b-da18935874c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:57.344714 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc4281-a82d-46e3-bb1b-da18935874c2 None admin] PATH_INFO: `/v3/projects/66ad66f40fde439f881153a21f8c2b44` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:57.349808 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc4281-a82d-46e3-bb1b-da18935874c2 None admin] RBAC: Authorizing `identity:get_project(project_id=66ad66f40fde439f881153a21f8c2b44)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:57.351639 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc4281-a82d-46e3-bb1b-da18935874c2 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:57.353544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 85/173] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:39:56 2022] GET /identity/v3/projects/66ad66f40fde439f881153a21f8c2b44 => generated 312 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:57.362890 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc4281-a82d-46e3-bb1b-da18935874c2 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:57.408694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94b2d8-821a-4ade-962c-94d2fd4a0886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:57.409322 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94b2d8-821a-4ade-962c-94d2fd4a0886 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:57.409605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94b2d8-821a-4ade-962c-94d2fd4a0886 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:57.409867 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94b2d8-821a-4ade-962c-94d2fd4a0886 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:57.433859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94b2d8-821a-4ade-962c-94d2fd4a0886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-fd94b2d8-821a-4ade-962c-94d2fd4a0886 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:57.439235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:57.440970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:57.442532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:57.442532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:57.442532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:57.442532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:57.442532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:57.442532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:57.442532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:57.442532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:57.442532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:57.442532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:57.442532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:39:57.442532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:39:57.445350 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 89/174] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:39:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:57.451376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94b2d8-821a-4ade-962c-94d2fd4a0886 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:57.592064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc4281-a82d-46e3-bb1b-da18935874c2 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:57.602137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94b2d8-821a-4ade-962c-94d2fd4a0886 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:57.645871 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3416a6-6fe0-446e-845b-31dc887b4613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:57.647226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3416a6-6fe0-446e-845b-31dc887b4613 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:57.647736 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3416a6-6fe0-446e-845b-31dc887b4613 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:57.647986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3416a6-6fe0-446e-845b-31dc887b4613 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:57.648805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3416a6-6fe0-446e-845b-31dc887b4613 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:57.649271 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3416a6-6fe0-446e-845b-31dc887b4613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:57.651072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3416a6-6fe0-446e-845b-31dc887b4613 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:57.667522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 90/175] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:57.670131 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3416a6-6fe0-446e-845b-31dc887b4613 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:57.685700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35480e13-66a3-46a6-a39c-82e5df99a34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:57.686367 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35480e13-66a3-46a6-a39c-82e5df99a34c None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:57.686639 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35480e13-66a3-46a6-a39c-82e5df99a34c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:57.686917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35480e13-66a3-46a6-a39c-82e5df99a34c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:57.687801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35480e13-66a3-46a6-a39c-82e5df99a34c None admin] RBAC: Adding query filter params (role.id=6a924f1d619643b3be92bab6fddb3eb2, scope.project.id=66ad66f40fde439f881153a21f8c2b44, user.id=843d8bc8740f48aaa93c0f79dc9687d0) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:57.688122 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35480e13-66a3-46a6-a39c-82e5df99a34c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:57.689807 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35480e13-66a3-46a6-a39c-82e5df99a34c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:57.695946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:57.705392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 86/176] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:39:57 2022] GET /identity/v3/role_assignments?user.id=843d8bc8740f48aaa93c0f79dc9687d0&scope.project.id=66ad66f40fde439f881153a21f8c2b44&role.id=6a924f1d619643b3be92bab6fddb3eb2 => generated 608 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:57.818923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3416a6-6fe0-446e-845b-31dc887b4613 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:57.871136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc633cd-e9c0-4e6e-9103-15b670b1bd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:57.871762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc633cd-e9c0-4e6e-9103-15b670b1bd8f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:57.872085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc633cd-e9c0-4e6e-9103-15b670b1bd8f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:57.872398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc633cd-e9c0-4e6e-9103-15b670b1bd8f None admin] PATH_INFO: `/v3/users/31dac45eb8764daf9a148a750fdc78a0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:57.874103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc633cd-e9c0-4e6e-9103-15b670b1bd8f None admin] RBAC: Authorizing `identity:get_user(user_id=31dac45eb8764daf9a148a750fdc78a0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:57.876028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc633cd-e9c0-4e6e-9103-15b670b1bd8f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:57.878302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 91/177] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:57 2022] GET /identity/v3/users/31dac45eb8764daf9a148a750fdc78a0 => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:57.884704 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35480e13-66a3-46a6-a39c-82e5df99a34c None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:57.909606 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:58.032782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c099f6-a3f4-4858-8b5e-1ad353a380ae None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:58.033195 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c099f6-a3f4-4858-8b5e-1ad353a380ae None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:58.033539 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c099f6-a3f4-4858-8b5e-1ad353a380ae None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:58.035292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 92/178] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:39:58.046678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307740c3-a37a-47dc-bc1a-df12a77d63f6 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:58.051489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307740c3-a37a-47dc-bc1a-df12a77d63f6 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:58.051489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307740c3-a37a-47dc-bc1a-df12a77d63f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:58.060481 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35480e13-66a3-46a6-a39c-82e5df99a34c None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:58.074465 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-307740c3-a37a-47dc-bc1a-df12a77d63f6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:58.074658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-307740c3-a37a-47dc-bc1a-df12a77d63f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:58.147982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7bea8-640e-4e9e-a4d8-8181dc3de7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:58.149485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7bea8-640e-4e9e-a4d8-8181dc3de7cc None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:58.149970 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7bea8-640e-4e9e-a4d8-8181dc3de7cc None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:58.156072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7bea8-640e-4e9e-a4d8-8181dc3de7cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:58.219533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 93/179] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:39:58.291766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7bea8-640e-4e9e-a4d8-8181dc3de7cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:58.293828 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7bea8-640e-4e9e-a4d8-8181dc3de7cc None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:58.295358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307740c3-a37a-47dc-bc1a-df12a77d63f6 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:58.299275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:58.373985 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:58.549280 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 87/180] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:39:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:39:58.560816 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7bea8-640e-4e9e-a4d8-8181dc3de7cc None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:58.670772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307740c3-a37a-47dc-bc1a-df12a77d63f6 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:58.794848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692af841-5e4c-4f1c-876b-14da81a96d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:58.795997 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692af841-5e4c-4f1c-876b-14da81a96d84 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:58.796455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692af841-5e4c-4f1c-876b-14da81a96d84 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:58.796883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692af841-5e4c-4f1c-876b-14da81a96d84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:58.835785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692af841-5e4c-4f1c-876b-14da81a96d84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-692af841-5e4c-4f1c-876b-14da81a96d84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:58.863503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:58.865910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:58.867498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:58.867498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:58.867498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:58.867498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:58.867498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:58.867498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:58.867498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:58.867498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:58.867498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:58.867498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:58.867498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:39:58.867498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:39:58.871903 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 94/181] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:39:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 577 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:39:58.883104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7bea8-640e-4e9e-a4d8-8181dc3de7cc None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:58.884941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692af841-5e4c-4f1c-876b-14da81a96d84 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:58.938934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:58.994670 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2666f9-3b1f-4697-94a0-716adb8fd361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:58.995908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2666f9-3b1f-4697-94a0-716adb8fd361 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:58.996430 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2666f9-3b1f-4697-94a0-716adb8fd361 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:58.996936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2666f9-3b1f-4697-94a0-716adb8fd361 None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:58.999951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2666f9-3b1f-4697-94a0-716adb8fd361 None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:59.003604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2666f9-3b1f-4697-94a0-716adb8fd361 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:59.008324 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 88/182] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:39:58 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:59.010671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642efa6-a8c5-461e-98cf-4ff3c14107ed None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:59.011130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642efa6-a8c5-461e-98cf-4ff3c14107ed None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:59.011618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642efa6-a8c5-461e-98cf-4ff3c14107ed None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:59.014447 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 89/183] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:59.022978 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642efa6-a8c5-461e-98cf-4ff3c14107ed None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:59.066045 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:59.128890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692af841-5e4c-4f1c-876b-14da81a96d84 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:59.186143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe2978-b76d-492f-98ac-ac547cc6e22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:59.187095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe2978-b76d-492f-98ac-ac547cc6e22a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:59.187617 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe2978-b76d-492f-98ac-ac547cc6e22a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:59.188038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe2978-b76d-492f-98ac-ac547cc6e22a None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:59.188975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe2978-b76d-492f-98ac-ac547cc6e22a None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:59.190177 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe2978-b76d-492f-98ac-ac547cc6e22a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:59.191997 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe2978-b76d-492f-98ac-ac547cc6e22a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:59.206124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 95/184] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:39:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:59.207893 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402b686-a25f-45d3-88ee-421f17a2c734 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:59.208978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402b686-a25f-45d3-88ee-421f17a2c734 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:59.209560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402b686-a25f-45d3-88ee-421f17a2c734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:59.235801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642efa6-a8c5-461e-98cf-4ff3c14107ed None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:59.237310 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-1402b686-a25f-45d3-88ee-421f17a2c734 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:59.239155 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1402b686-a25f-45d3-88ee-421f17a2c734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:59.298445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bbcd2-b1c7-42f7-8d13-d7a019148f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:59.299708 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bbcd2-b1c7-42f7-8d13-d7a019148f5c None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:59.300049 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bbcd2-b1c7-42f7-8d13-d7a019148f5c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:59.300337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bbcd2-b1c7-42f7-8d13-d7a019148f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:59.301202 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bbcd2-b1c7-42f7-8d13-d7a019148f5c None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=d0c721e2529541e5966d3c21426e447a, user.id=31dac45eb8764daf9a148a750fdc78a0) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:39:59.301510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bbcd2-b1c7-42f7-8d13-d7a019148f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:59.303242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bbcd2-b1c7-42f7-8d13-d7a019148f5c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:59.311910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 90/185] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:39:59 2022] GET /identity/v3/role_assignments?user.id=31dac45eb8764daf9a148a750fdc78a0&scope.project.id=d0c721e2529541e5966d3c21426e447a&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:39:59.322159 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829162a-b9e9-407a-b6a2-c30536360706 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:59.323134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829162a-b9e9-407a-b6a2-c30536360706 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:59.323960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829162a-b9e9-407a-b6a2-c30536360706 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:59.327020 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 91/186] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:59 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:59.328935 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829162a-b9e9-407a-b6a2-c30536360706 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:59.404325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 96/187] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:59.404325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab19427-14a5-44a4-b2ea-db496f1e7767 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:59.404325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab19427-14a5-44a4-b2ea-db496f1e7767 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:59.404325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab19427-14a5-44a4-b2ea-db496f1e7767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:59.457746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-dab19427-14a5-44a4-b2ea-db496f1e7767 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:59.461005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-dab19427-14a5-44a4-b2ea-db496f1e7767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:59.483000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:59.582770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:59.599734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829162a-b9e9-407a-b6a2-c30536360706 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:59.610008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 97/188] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:39:59.611222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab19427-14a5-44a4-b2ea-db496f1e7767 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:59.669824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457367ee-3533-455e-9615-e588b75e4573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:59.670875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457367ee-3533-455e-9615-e588b75e4573 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:59.671378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457367ee-3533-455e-9615-e588b75e4573 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:59.671896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457367ee-3533-455e-9615-e588b75e4573 None admin] PATH_INFO: `/v3/users/d63c97bc953b43dc83b6633361d498df` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:59.701762 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457367ee-3533-455e-9615-e588b75e4573 None admin] RBAC: Authorizing `identity:get_user(user_id=d63c97bc953b43dc83b6633361d498df)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:59.704233 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457367ee-3533-455e-9615-e588b75e4573 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:59.707489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 92/189] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:39:59 2022] GET /identity/v3/users/d63c97bc953b43dc83b6633361d498df => generated 283 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:59.709754 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982d555-ac4a-4aa4-a436-20139959cae1 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:59.710240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982d555-ac4a-4aa4-a436-20139959cae1 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:59.710731 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982d555-ac4a-4aa4-a436-20139959cae1 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:59.713275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 93/190] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:39:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:39:59.714626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982d555-ac4a-4aa4-a436-20139959cae1 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:59.806465 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab19427-14a5-44a4-b2ea-db496f1e7767 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:59.874031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da405a-20f5-4419-aacd-bc9df83b5c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:59.875172 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da405a-20f5-4419-aacd-bc9df83b5c13 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:59.875716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da405a-20f5-4419-aacd-bc9df83b5c13 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:59.876220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da405a-20f5-4419-aacd-bc9df83b5c13 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:59.876590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982d555-ac4a-4aa4-a436-20139959cae1 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:39:59.888969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:39:59.899080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da405a-20f5-4419-aacd-bc9df83b5c13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-b1da405a-20f5-4419-aacd-bc9df83b5c13 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:39:59.908760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:39:59.910015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:39:59.910814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:39:59.910814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:39:59.910814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:39:59.910814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:39:59.910814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:39:59.910814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:39:59.910814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:39:59.910814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:39:59.910814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:39:59.910814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:39:59.910814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:39:59.910814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:39:59.912147 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 98/191] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:39:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:39:59.914054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da405a-20f5-4419-aacd-bc9df83b5c13 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:39:59.925493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e3ea9-fdfa-4e50-93fd-85bc373e4801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:39:59.926459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e3ea9-fdfa-4e50-93fd-85bc373e4801 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:59.927002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e3ea9-fdfa-4e50-93fd-85bc373e4801 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:59.927977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e3ea9-fdfa-4e50-93fd-85bc373e4801 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:59.930123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e3ea9-fdfa-4e50-93fd-85bc373e4801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:39:59.932938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e3ea9-fdfa-4e50-93fd-85bc373e4801 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:39:59.936487 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 94/192] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:39:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:39:59.938709 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f450a-2244-4378-b2b2-4fe3aed10025 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:39:59.939732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f450a-2244-4378-b2b2-4fe3aed10025 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:39:59.940306 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f450a-2244-4378-b2b2-4fe3aed10025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:39:59.970870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-d60f450a-2244-4378-b2b2-4fe3aed10025 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:39:59.972801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d60f450a-2244-4378-b2b2-4fe3aed10025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:39:59.978216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:00.048517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da405a-20f5-4419-aacd-bc9df83b5c13 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:00.091342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65014814-e8d6-4abd-9308-a0b0d7dc134d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:00.092075 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014814-e8d6-4abd-9308-a0b0d7dc134d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:00.092380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014814-e8d6-4abd-9308-a0b0d7dc134d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:00.092731 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014814-e8d6-4abd-9308-a0b0d7dc134d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:00.098401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 95/193] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:00.108009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f450a-2244-4378-b2b2-4fe3aed10025 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:00.194139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65014814-e8d6-4abd-9308-a0b0d7dc134d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:00.196156 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65014814-e8d6-4abd-9308-a0b0d7dc134d None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:00.254147 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f450a-2244-4378-b2b2-4fe3aed10025 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:00.259833 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:00.266252 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:00.312630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57622661-2c85-47ff-a135-711b3c6fdaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:00.313341 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57622661-2c85-47ff-a135-711b3c6fdaea None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:00.313662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57622661-2c85-47ff-a135-711b3c6fdaea None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:00.313980 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57622661-2c85-47ff-a135-711b3c6fdaea None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:00.314779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57622661-2c85-47ff-a135-711b3c6fdaea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:00.315216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57622661-2c85-47ff-a135-711b3c6fdaea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:00.316947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57622661-2c85-47ff-a135-711b3c6fdaea None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:00.328137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 96/194] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:40:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:00.329501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57622661-2c85-47ff-a135-711b3c6fdaea None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:00.394840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 99/195] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:39:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:00.397378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65014814-e8d6-4abd-9308-a0b0d7dc134d None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:00.494943 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57622661-2c85-47ff-a135-711b3c6fdaea None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:00.534635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65014814-e8d6-4abd-9308-a0b0d7dc134d None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:00.541302 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748758b-cd1a-48fb-88b5-aafddc7ca72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:00.542020 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748758b-cd1a-48fb-88b5-aafddc7ca72d None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:00.542362 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748758b-cd1a-48fb-88b5-aafddc7ca72d None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:00.542843 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748758b-cd1a-48fb-88b5-aafddc7ca72d None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:00.543681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748758b-cd1a-48fb-88b5-aafddc7ca72d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:00.545371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748758b-cd1a-48fb-88b5-aafddc7ca72d None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:00.581933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e08f9-5375-463a-8086-45a565997091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:00.582983 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e08f9-5375-463a-8086-45a565997091 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:00.584025 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e08f9-5375-463a-8086-45a565997091 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:00.584582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e08f9-5375-463a-8086-45a565997091 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:00.600067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e08f9-5375-463a-8086-45a565997091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-825e08f9-5375-463a-8086-45a565997091 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:00.605576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:00.607348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:00.609002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:00.609002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:00.609002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:00.609002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:00.609002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:00.609002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:00.609002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:00.609002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:00.609002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:00.609002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:00.609002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:40:00.609002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:00.609002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 100/196] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:00.609002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e08f9-5375-463a-8086-45a565997091 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:00.621228 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:00.653703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 97/197] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:40:00 2022] POST /identity/v3/users => generated 297 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:40:00.655300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748758b-cd1a-48fb-88b5-aafddc7ca72d None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:00.705764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:00.770130 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e08f9-5375-463a-8086-45a565997091 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:00.838060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7f379-fbda-45af-8c72-2e824353d1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:00.838722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7f379-fbda-45af-8c72-2e824353d1fa None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:00.839033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7f379-fbda-45af-8c72-2e824353d1fa None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:00.839320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7f379-fbda-45af-8c72-2e824353d1fa None admin] PATH_INFO: `/v3/users/d63c97bc953b43dc83b6633361d498df` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:00.839634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748758b-cd1a-48fb-88b5-aafddc7ca72d None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:00.840901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be7f379-fbda-45af-8c72-2e824353d1fa None admin] RBAC: Authorizing `identity:get_user(user_id=d63c97bc953b43dc83b6633361d498df)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:00.842565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be7f379-fbda-45af-8c72-2e824353d1fa None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:00.844639 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 101/198] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:00 2022] GET /identity/v3/users/d63c97bc953b43dc83b6633361d498df => generated 283 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:00.845343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7f379-fbda-45af-8c72-2e824353d1fa None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:00.905736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7de990-6c95-4a6f-bbf2-8303cb5e0f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:00.907875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de990-6c95-4a6f-bbf2-8303cb5e0f47 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:00.908392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de990-6c95-4a6f-bbf2-8303cb5e0f47 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:00.908901 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de990-6c95-4a6f-bbf2-8303cb5e0f47 None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:00.911341 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7de990-6c95-4a6f-bbf2-8303cb5e0f47 None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:00.914986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7de990-6c95-4a6f-bbf2-8303cb5e0f47 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:00.918498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 98/199] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:00 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:00.920071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7de990-6c95-4a6f-bbf2-8303cb5e0f47 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:00.979103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:01.023034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be7f379-fbda-45af-8c72-2e824353d1fa None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:01.112297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056cf22-16f3-4652-ae8f-84a26d29d949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:01.113950 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056cf22-16f3-4652-ae8f-84a26d29d949 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:01.114511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056cf22-16f3-4652-ae8f-84a26d29d949 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:01.115564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056cf22-16f3-4652-ae8f-84a26d29d949 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:01.116914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056cf22-16f3-4652-ae8f-84a26d29d949 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:01.117442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056cf22-16f3-4652-ae8f-84a26d29d949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:01.117587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:01.120669 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056cf22-16f3-4652-ae8f-84a26d29d949 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:01.139208 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 102/200] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:01.140325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7de990-6c95-4a6f-bbf2-8303cb5e0f47 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:01.141061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056cf22-16f3-4652-ae8f-84a26d29d949 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:01.224598 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982d36d-1066-47da-9536-b1af102e2945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:01.226018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982d36d-1066-47da-9536-b1af102e2945 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:01.226869 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982d36d-1066-47da-9536-b1af102e2945 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:01.228165 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982d36d-1066-47da-9536-b1af102e2945 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:01.326204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056cf22-16f3-4652-ae8f-84a26d29d949 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:01.374008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6dc934-4b43-47d8-babc-ac1d2960cd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:01.374754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dc934-4b43-47d8-babc-ac1d2960cd44 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:01.375041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dc934-4b43-47d8-babc-ac1d2960cd44 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:01.375494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dc934-4b43-47d8-babc-ac1d2960cd44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:01.376507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6dc934-4b43-47d8-babc-ac1d2960cd44 None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=21ad3a8335ad492d858142d787614be0, user.id=d63c97bc953b43dc83b6633361d498df) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:01.376858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6dc934-4b43-47d8-babc-ac1d2960cd44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:01.378501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6dc934-4b43-47d8-babc-ac1d2960cd44 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:01.382851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:01.386663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 103/201] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:40:01 2022] GET /identity/v3/role_assignments?user.id=d63c97bc953b43dc83b6633361d498df&scope.project.id=21ad3a8335ad492d858142d787614be0&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:01.388063 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6dc934-4b43-47d8-babc-ac1d2960cd44 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:01.404073 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982d36d-1066-47da-9536-b1af102e2945 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:01.407717 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982d36d-1066-47da-9536-b1af102e2945 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:01.559322 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6dc934-4b43-47d8-babc-ac1d2960cd44 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:01.590799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:01.605345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 99/202] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 686 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:01.611310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982d36d-1066-47da-9536-b1af102e2945 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:01.655158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe628a2-6fab-4d25-bad5-3deb654b2459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:01.656784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe628a2-6fab-4d25-bad5-3deb654b2459 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:01.660494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe628a2-6fab-4d25-bad5-3deb654b2459 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:01.662109 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe628a2-6fab-4d25-bad5-3deb654b2459 None admin] PATH_INFO: `/v3/users/84fbc773412c4d25a3aedf5914c636fc` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:01.668623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe628a2-6fab-4d25-bad5-3deb654b2459 None admin] RBAC: Authorizing `identity:get_user(user_id=84fbc773412c4d25a3aedf5914c636fc)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:01.673879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe628a2-6fab-4d25-bad5-3deb654b2459 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:01.678924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 104/203] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:01 2022] GET /identity/v3/users/84fbc773412c4d25a3aedf5914c636fc => generated 289 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:01.684748 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe628a2-6fab-4d25-bad5-3deb654b2459 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:01.853257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:01.919665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982d36d-1066-47da-9536-b1af102e2945 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:01.933402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe628a2-6fab-4d25-bad5-3deb654b2459 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:02.006404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ddcc0a-5bc0-43b8-9b56-8566bc06cbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:02.010679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddcc0a-5bc0-43b8-9b56-8566bc06cbdb None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:02.014280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddcc0a-5bc0-43b8-9b56-8566bc06cbdb None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:02.015678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddcc0a-5bc0-43b8-9b56-8566bc06cbdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:02.055624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28ba55-3506-444e-b1a7-17b1af413131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:02.064873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28ba55-3506-444e-b1a7-17b1af413131 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:02.065376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28ba55-3506-444e-b1a7-17b1af413131 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:02.065877 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28ba55-3506-444e-b1a7-17b1af413131 None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:02.068803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28ba55-3506-444e-b1a7-17b1af413131 None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:02.076175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28ba55-3506-444e-b1a7-17b1af413131 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:02.079682 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 100/204] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:01 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:02.088023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28ba55-3506-444e-b1a7-17b1af413131 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:02.172199 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ddcc0a-5bc0-43b8-9b56-8566bc06cbdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:02.175136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ddcc0a-5bc0-43b8-9b56-8566bc06cbdb None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:02.327221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:02.360748 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:02.375953 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 105/205] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 14:40:02.387055 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ddcc0a-5bc0-43b8-9b56-8566bc06cbdb None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:02.500007 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28ba55-3506-444e-b1a7-17b1af413131 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:02.608519 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bb14d-56c8-46d6-9f67-0e8a294c1612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:02.609613 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bb14d-56c8-46d6-9f67-0e8a294c1612 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:02.610110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bb14d-56c8-46d6-9f67-0e8a294c1612 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:02.610601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bb14d-56c8-46d6-9f67-0e8a294c1612 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:02.612173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bb14d-56c8-46d6-9f67-0e8a294c1612 None admin] RBAC: Adding query filter params (role.id=eff5cca513be4ed5a9b001c0559fb476, scope.project.id=21ad3a8335ad492d858142d787614be0, user.id=d63c97bc953b43dc83b6633361d498df) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:02.612691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bb14d-56c8-46d6-9f67-0e8a294c1612 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:02.616342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bb14d-56c8-46d6-9f67-0e8a294c1612 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:02.644035 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 101/206] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:40:02 2022] GET /identity/v3/role_assignments?user.id=d63c97bc953b43dc83b6633361d498df&scope.project.id=21ad3a8335ad492d858142d787614be0&role.id=eff5cca513be4ed5a9b001c0559fb476 => generated 259 bytes in 557 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:02.696638 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ddcc0a-5bc0-43b8-9b56-8566bc06cbdb None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:02.784537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430e35d-aa0f-4b19-b8bb-5ccb896aa908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:02.785351 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430e35d-aa0f-4b19-b8bb-5ccb896aa908 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:02.785658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430e35d-aa0f-4b19-b8bb-5ccb896aa908 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:02.785958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430e35d-aa0f-4b19-b8bb-5ccb896aa908 None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:02.788432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430e35d-aa0f-4b19-b8bb-5ccb896aa908 None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:02.789999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430e35d-aa0f-4b19-b8bb-5ccb896aa908 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:02.792258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 106/207] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:02 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:02.799710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bb14d-56c8-46d6-9f67-0e8a294c1612 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:03.115846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bb14d-56c8-46d6-9f67-0e8a294c1612 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:03.166933 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:03.197109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5b08c-ef75-4eb2-8f07-98786f7d9db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:03.198188 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5b08c-ef75-4eb2-8f07-98786f7d9db6 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:03.198676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5b08c-ef75-4eb2-8f07-98786f7d9db6 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:03.199138 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5b08c-ef75-4eb2-8f07-98786f7d9db6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:03.200614 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5b08c-ef75-4eb2-8f07-98786f7d9db6 None admin] RBAC: Adding query filter params (role.id=cf7c52610e954f74a30feba325390387, scope.project.id=d0c721e2529541e5966d3c21426e447a, user.id=84fbc773412c4d25a3aedf5914c636fc) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:03.201348 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5b08c-ef75-4eb2-8f07-98786f7d9db6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:03.204216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5b08c-ef75-4eb2-8f07-98786f7d9db6 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:03.225073 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 102/208] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:40:02 2022] GET /identity/v3/role_assignments?user.id=84fbc773412c4d25a3aedf5914c636fc&scope.project.id=d0c721e2529541e5966d3c21426e447a&role.id=cf7c52610e954f74a30feba325390387 => generated 608 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:03.543895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dfc5e-0ff4-4bf2-a1ab-8a987fb8c732 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:03.544640 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dfc5e-0ff4-4bf2-a1ab-8a987fb8c732 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:03.544640 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dfc5e-0ff4-4bf2-a1ab-8a987fb8c732 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:03.546071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 107/209] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:03.562922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd4b5b-d3c1-463c-b49a-3f5316ed9767 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:03.563908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd4b5b-d3c1-463c-b49a-3f5316ed9767 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:03.564681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd4b5b-d3c1-463c-b49a-3f5316ed9767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:03.611345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-56bd4b5b-d3c1-463c-b49a-3f5316ed9767 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:03.613804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-56bd4b5b-d3c1-463c-b49a-3f5316ed9767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:03.787068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 103/210] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:03.831916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dfc5e-0ff4-4bf2-a1ab-8a987fb8c732 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:03.902129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:04.037195 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dfc5e-0ff4-4bf2-a1ab-8a987fb8c732 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:04.111550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060eccb-f966-4072-b19b-14fa7f494830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:04.112178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060eccb-f966-4072-b19b-14fa7f494830 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:04.112543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060eccb-f966-4072-b19b-14fa7f494830 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:04.112906 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060eccb-f966-4072-b19b-14fa7f494830 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:04.128111 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0060eccb-f966-4072-b19b-14fa7f494830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-0060eccb-f966-4072-b19b-14fa7f494830 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:04.135050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:04.136929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:04.138560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:04.138560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:04.138560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:04.138560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:04.138560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:04.138560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:04.138560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:04.138560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:04.138560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:04.138560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:04.138560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:04.138560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:04.138560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 108/211] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:04.147500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd4b5b-d3c1-463c-b49a-3f5316ed9767 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:04.208734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b3b36-9658-4a88-8494-8968d0b5cdc1 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:04.209213 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b3b36-9658-4a88-8494-8968d0b5cdc1 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:04.209656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b3b36-9658-4a88-8494-8968d0b5cdc1 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:04.212309 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 109/212] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:40:04.221305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1264e-80b6-4eda-9a66-122cff4fcad6 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:04.222107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1264e-80b6-4eda-9a66-122cff4fcad6 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:04.222769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1264e-80b6-4eda-9a66-122cff4fcad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:04.268777 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-4df1264e-80b6-4eda-9a66-122cff4fcad6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:04.270124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4df1264e-80b6-4eda-9a66-122cff4fcad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:04.325712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd4b5b-d3c1-463c-b49a-3f5316ed9767 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:04.349748 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:04.369997 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc951e-47e3-45bd-8256-50dc3bc21d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:04.370611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc951e-47e3-45bd-8256-50dc3bc21d63 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:04.370907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc951e-47e3-45bd-8256-50dc3bc21d63 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:04.371178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc951e-47e3-45bd-8256-50dc3bc21d63 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:04.371974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc951e-47e3-45bd-8256-50dc3bc21d63 None admin] RBAC: Adding query filter params (name=member) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:04.372330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc951e-47e3-45bd-8256-50dc3bc21d63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:04.373896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc951e-47e3-45bd-8256-50dc3bc21d63 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:04.387497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 110/213] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:04.387497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9980bd-3e92-491e-bc45-c0811616151f None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:04.387497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9980bd-3e92-491e-bc45-c0811616151f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:04.387497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9980bd-3e92-491e-bc45-c0811616151f None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:04.387497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 111/214] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:04.393478 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 104/215] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:04.396924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a6051-93b7-4729-9f42-b2c22780cbd7 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:04.397395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a6051-93b7-4729-9f42-b2c22780cbd7 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:04.397883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a6051-93b7-4729-9f42-b2c22780cbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:04.402077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc951e-47e3-45bd-8256-50dc3bc21d63 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:04.434755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-f94a6051-93b7-4729-9f42-b2c22780cbd7 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:04.436413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f94a6051-93b7-4729-9f42-b2c22780cbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:04.469117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:04.532756 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 112/216] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:40:04.534306 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a6051-93b7-4729-9f42-b2c22780cbd7 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:04.566259 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc951e-47e3-45bd-8256-50dc3bc21d63 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:04.634576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da93d3-01cb-4ef5-9b3c-02d1f25e1184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:04.635870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da93d3-01cb-4ef5-9b3c-02d1f25e1184 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:04.636527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da93d3-01cb-4ef5-9b3c-02d1f25e1184 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:04.637025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da93d3-01cb-4ef5-9b3c-02d1f25e1184 None admin] PATH_INFO: `/v3/users/4025368a8cf44d6ebecb90392d7b799a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:04.666638 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da93d3-01cb-4ef5-9b3c-02d1f25e1184 None admin] RBAC: Authorizing `identity:get_user(user_id=4025368a8cf44d6ebecb90392d7b799a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:04.668224 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da93d3-01cb-4ef5-9b3c-02d1f25e1184 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:04.670042 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 105/217] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:04 2022] GET /identity/v3/users/4025368a8cf44d6ebecb90392d7b799a => generated 297 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:04.670851 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da93d3-01cb-4ef5-9b3c-02d1f25e1184 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:04.703721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a6051-93b7-4729-9f42-b2c22780cbd7 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:04.729783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:04.758704 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e74ad-5229-48d3-aa41-ef250c5292e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:04.759587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e74ad-5229-48d3-aa41-ef250c5292e4 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:04.760075 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e74ad-5229-48d3-aa41-ef250c5292e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:04.760495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e74ad-5229-48d3-aa41-ef250c5292e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:04.762504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e74ad-5229-48d3-aa41-ef250c5292e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:04.764779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e74ad-5229-48d3-aa41-ef250c5292e4 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:04.767205 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 113/218] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:40:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:04.768261 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e74ad-5229-48d3-aa41-ef250c5292e4 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:04.804739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:04.814268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da93d3-01cb-4ef5-9b3c-02d1f25e1184 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:04.866407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f4956-8b41-4f7a-bdb6-d520955a7bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:04.867055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f4956-8b41-4f7a-bdb6-d520955a7bb5 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:04.867346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f4956-8b41-4f7a-bdb6-d520955a7bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:04.867657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f4956-8b41-4f7a-bdb6-d520955a7bb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:04.881441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f4956-8b41-4f7a-bdb6-d520955a7bb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-ff5f4956-8b41-4f7a-bdb6-d520955a7bb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:04.887157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:04.889180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:04.890873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:04.890873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:04.890873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:04.890873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:04.890873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:04.890873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:04.890873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:04.890873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:04.890873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:04.890873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:04.890873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:40:04.890873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:04.890873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 106/219] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:40:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:04.896372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f4956-8b41-4f7a-bdb6-d520955a7bb5 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:04.924993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e74ad-5229-48d3-aa41-ef250c5292e4 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:04.984457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9a9b1-19c9-48ef-8e64-cb2c1706c7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:04.985524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a9b1-19c9-48ef-8e64-cb2c1706c7b6 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:04.986009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a9b1-19c9-48ef-8e64-cb2c1706c7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:04.986491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a9b1-19c9-48ef-8e64-cb2c1706c7b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:05.039346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:05.044586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f4956-8b41-4f7a-bdb6-d520955a7bb5 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:05.089807 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61763663-d277-4d1f-81c5-10dc388da0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:05.090833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61763663-d277-4d1f-81c5-10dc388da0cc None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:05.091137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61763663-d277-4d1f-81c5-10dc388da0cc None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:05.091544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61763663-d277-4d1f-81c5-10dc388da0cc None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:05.092272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61763663-d277-4d1f-81c5-10dc388da0cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:05.094227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61763663-d277-4d1f-81c5-10dc388da0cc None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:05.105254 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9a9b1-19c9-48ef-8e64-cb2c1706c7b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:05.107600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9a9b1-19c9-48ef-8e64-cb2c1706c7b6 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:05.140765 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:05.242183 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 114/220] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:05.243220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9a9b1-19c9-48ef-8e64-cb2c1706c7b6 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:05.268657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 107/221] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:40:04 2022] POST /identity/v3/users => generated 297 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:40:05.271827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61763663-d277-4d1f-81c5-10dc388da0cc None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:05.470104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9a9b1-19c9-48ef-8e64-cb2c1706c7b6 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:05.514107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:05.517492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61763663-d277-4d1f-81c5-10dc388da0cc None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:05.536034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2320c-b111-4e49-b603-db98b0f9a895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:05.537623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2320c-b111-4e49-b603-db98b0f9a895 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:05.538119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2320c-b111-4e49-b603-db98b0f9a895 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:05.538598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2320c-b111-4e49-b603-db98b0f9a895 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:05.540154 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2320c-b111-4e49-b603-db98b0f9a895 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:05.540713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2320c-b111-4e49-b603-db98b0f9a895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:05.543396 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2320c-b111-4e49-b603-db98b0f9a895 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:05.565319 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 115/222] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:05.573034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2320c-b111-4e49-b603-db98b0f9a895 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:05.586815 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d06014-51b1-49d7-9d47-02f9762bf81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:05.587917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d06014-51b1-49d7-9d47-02f9762bf81a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:05.588288 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d06014-51b1-49d7-9d47-02f9762bf81a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:05.590088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d06014-51b1-49d7-9d47-02f9762bf81a None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:05.591956 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d06014-51b1-49d7-9d47-02f9762bf81a None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:05.593980 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d06014-51b1-49d7-9d47-02f9762bf81a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:05.597843 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 108/223] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:05 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:05.606639 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d06014-51b1-49d7-9d47-02f9762bf81a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:05.659391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:05.758562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2320c-b111-4e49-b603-db98b0f9a895 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:05.803947 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d2410-d16d-4fe1-80c8-c9a2835c5a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:05.804963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d2410-d16d-4fe1-80c8-c9a2835c5a23 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:05.805314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d2410-d16d-4fe1-80c8-c9a2835c5a23 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:05.805714 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d2410-d16d-4fe1-80c8-c9a2835c5a23 None admin] PATH_INFO: `/v3/users/d63c97bc953b43dc83b6633361d498df` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:05.807359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d2410-d16d-4fe1-80c8-c9a2835c5a23 None admin] RBAC: Authorizing `identity:get_user(user_id=d63c97bc953b43dc83b6633361d498df)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:05.809072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d2410-d16d-4fe1-80c8-c9a2835c5a23 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:05.811511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 116/224] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:05 2022] GET /identity/v3/users/d63c97bc953b43dc83b6633361d498df => generated 283 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:05.818298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d06014-51b1-49d7-9d47-02f9762bf81a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:05.821287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d2410-d16d-4fe1-80c8-c9a2835c5a23 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:05.884123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7614f-6a5c-41ee-a8cb-de7e35e98ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:05.885152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7614f-6a5c-41ee-a8cb-de7e35e98ce0 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:05.885621 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7614f-6a5c-41ee-a8cb-de7e35e98ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:05.886119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7614f-6a5c-41ee-a8cb-de7e35e98ce0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:05.887170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7614f-6a5c-41ee-a8cb-de7e35e98ce0 None admin] RBAC: Adding query filter params (role.id=6a924f1d619643b3be92bab6fddb3eb2, scope.project.id=21ad3a8335ad492d858142d787614be0, user.id=4025368a8cf44d6ebecb90392d7b799a) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:05.887734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7614f-6a5c-41ee-a8cb-de7e35e98ce0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:05.890248 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7614f-6a5c-41ee-a8cb-de7e35e98ce0 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:05.902146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 109/225] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:40:05 2022] GET /identity/v3/role_assignments?user.id=4025368a8cf44d6ebecb90392d7b799a&scope.project.id=21ad3a8335ad492d858142d787614be0&role.id=6a924f1d619643b3be92bab6fddb3eb2 => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:05.920496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:05.976348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d2410-d16d-4fe1-80c8-c9a2835c5a23 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:06.024585 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5142d-74fb-42f1-9294-476aaf900eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:06.025715 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5142d-74fb-42f1-9294-476aaf900eac None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:06.026171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5142d-74fb-42f1-9294-476aaf900eac None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:06.026550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5142d-74fb-42f1-9294-476aaf900eac None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:06.028181 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5142d-74fb-42f1-9294-476aaf900eac None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:06.029982 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5142d-74fb-42f1-9294-476aaf900eac None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:06.032818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 117/226] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:05 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:06.044103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7614f-6a5c-41ee-a8cb-de7e35e98ce0 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:06.280659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:06.323276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7614f-6a5c-41ee-a8cb-de7e35e98ce0 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:06.418333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d458782-466e-4b6e-bf2c-f274e54a4cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:06.419591 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d458782-466e-4b6e-bf2c-f274e54a4cba None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:06.420092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d458782-466e-4b6e-bf2c-f274e54a4cba None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:06.420579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d458782-466e-4b6e-bf2c-f274e54a4cba None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0/users/d63c97bc953b43dc83b6633361d498df/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:06.424511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d458782-466e-4b6e-bf2c-f274e54a4cba None admin] RBAC: Authorizing `identity:create_grant(project_id=21ad3a8335ad492d858142d787614be0, user_id=d63c97bc953b43dc83b6633361d498df, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:06.429652 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d458782-466e-4b6e-bf2c-f274e54a4cba None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:06.451712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519939f-e0d5-4040-8a66-aa33c10ca86f None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:06.452186 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519939f-e0d5-4040-8a66-aa33c10ca86f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:06.452688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519939f-e0d5-4040-8a66-aa33c10ca86f None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:06.455506 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 118/227] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:40:06.462182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe237f-3271-444a-8c71-f5d3c1862ae1 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:06.462665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe237f-3271-444a-8c71-f5d3c1862ae1 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:06.463139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe237f-3271-444a-8c71-f5d3c1862ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:06.497721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-0cbe237f-3271-444a-8c71-f5d3c1862ae1 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:06.499848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-0cbe237f-3271-444a-8c71-f5d3c1862ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:06.513917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 110/228] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:40:06 2022] PUT /identity/v3/projects/21ad3a8335ad492d858142d787614be0/users/d63c97bc953b43dc83b6633361d498df/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:40:06.662781 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 119/229] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:40:06.699341 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d458782-466e-4b6e-bf2c-f274e54a4cba None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:06.879487 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d458782-466e-4b6e-bf2c-f274e54a4cba None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:06.926855 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ee104-6155-48c0-9b40-41c64885f542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:06.927960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ee104-6155-48c0-9b40-41c64885f542 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:06.928471 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ee104-6155-48c0-9b40-41c64885f542 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:06.928978 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ee104-6155-48c0-9b40-41c64885f542 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:06.962302 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ee104-6155-48c0-9b40-41c64885f542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-ba3ee104-6155-48c0-9b40-41c64885f542 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:06.973646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:06.975083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:06.976060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:06.976060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:06.976060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:06.976060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:06.976060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:06.976060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:06.976060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:06.976060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:06.976060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:06.976060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:06.976060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:06.976060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:06.976712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 111/230] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:40:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:06.984174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe237f-3271-444a-8c71-f5d3c1862ae1 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:07.113697 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:07.249552 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe237f-3271-444a-8c71-f5d3c1862ae1 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:07.347923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92d507-07f2-4c7c-b986-d1292f6449fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:07.349610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92d507-07f2-4c7c-b986-d1292f6449fb None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:07.350125 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92d507-07f2-4c7c-b986-d1292f6449fb None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:07.350602 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92d507-07f2-4c7c-b986-d1292f6449fb None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:07.352275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92d507-07f2-4c7c-b986-d1292f6449fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:07.352820 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92d507-07f2-4c7c-b986-d1292f6449fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:07.355451 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92d507-07f2-4c7c-b986-d1292f6449fb None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:07.383674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 120/231] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:40:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:07.390629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ee104-6155-48c0-9b40-41c64885f542 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:07.412110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:07.650908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ee104-6155-48c0-9b40-41c64885f542 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:07.730694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fc09d-8d1e-4a71-b582-3a5c3353f676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:07.731818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fc09d-8d1e-4a71-b582-3a5c3353f676 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:07.732527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fc09d-8d1e-4a71-b582-3a5c3353f676 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:07.733086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fc09d-8d1e-4a71-b582-3a5c3353f676 None admin] PATH_INFO: `/v3/users/d63c97bc953b43dc83b6633361d498df` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:07.735553 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fc09d-8d1e-4a71-b582-3a5c3353f676 None admin] RBAC: Authorizing `identity:get_user(user_id=d63c97bc953b43dc83b6633361d498df)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:07.738161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fc09d-8d1e-4a71-b582-3a5c3353f676 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:07.741608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 112/232] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:07 2022] GET /identity/v3/users/d63c97bc953b43dc83b6633361d498df => generated 283 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:07.750533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92d507-07f2-4c7c-b986-d1292f6449fb None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:08.023056 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92d507-07f2-4c7c-b986-d1292f6449fb None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:08.056136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:08.090557 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288a369-e331-4c47-8c6a-11db2401eb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:08.092187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288a369-e331-4c47-8c6a-11db2401eb0d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:08.092978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288a369-e331-4c47-8c6a-11db2401eb0d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:08.093784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288a369-e331-4c47-8c6a-11db2401eb0d None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:08.096548 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288a369-e331-4c47-8c6a-11db2401eb0d None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:08.105781 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288a369-e331-4c47-8c6a-11db2401eb0d None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:08.109576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 121/233] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:07 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:08.120115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fc09d-8d1e-4a71-b582-3a5c3353f676 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:08.321610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:08.372740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fc09d-8d1e-4a71-b582-3a5c3353f676 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:08.455014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8159084-df31-4c59-ba3b-43eca5024072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:08.456241 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8159084-df31-4c59-ba3b-43eca5024072 None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:08.456719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8159084-df31-4c59-ba3b-43eca5024072 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:08.457183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8159084-df31-4c59-ba3b-43eca5024072 None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0/users/d63c97bc953b43dc83b6633361d498df/roles/eff5cca513be4ed5a9b001c0559fb476` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:08.466211 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8159084-df31-4c59-ba3b-43eca5024072 None admin] RBAC: Authorizing `identity:create_grant(project_id=21ad3a8335ad492d858142d787614be0, user_id=d63c97bc953b43dc83b6633361d498df, role_id=eff5cca513be4ed5a9b001c0559fb476)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:08.469128 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8159084-df31-4c59-ba3b-43eca5024072 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:08.604742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dc12b-0192-40dd-ab52-17fb76f2e730 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:08.604966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dc12b-0192-40dd-ab52-17fb76f2e730 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:08.605298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dc12b-0192-40dd-ab52-17fb76f2e730 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:08.607120 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 122/234] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:08.611799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 113/235] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:40:08 2022] PUT /identity/v3/projects/21ad3a8335ad492d858142d787614be0/users/d63c97bc953b43dc83b6633361d498df/roles/eff5cca513be4ed5a9b001c0559fb476 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:40:08.613035 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28adbe-cd3d-4258-a46d-d9e9312bc3fc None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:08.613424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28adbe-cd3d-4258-a46d-d9e9312bc3fc None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:08.613811 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28adbe-cd3d-4258-a46d-d9e9312bc3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:08.649623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-4f28adbe-cd3d-4258-a46d-d9e9312bc3fc None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:08.650991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4f28adbe-cd3d-4258-a46d-d9e9312bc3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:08.775790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 123/236] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:08.817786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8159084-df31-4c59-ba3b-43eca5024072 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:09.047296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8159084-df31-4c59-ba3b-43eca5024072 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:09.127352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b5bc1-b6af-43ea-ab05-2af6256a1119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:09.128499 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b5bc1-b6af-43ea-ab05-2af6256a1119 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:09.129001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b5bc1-b6af-43ea-ab05-2af6256a1119 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:09.129488 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b5bc1-b6af-43ea-ab05-2af6256a1119 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:09.137144 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b5bc1-b6af-43ea-ab05-2af6256a1119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:09.140201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b5bc1-b6af-43ea-ab05-2af6256a1119 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:09.144109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 114/237] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:40:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:09.150388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28adbe-cd3d-4258-a46d-d9e9312bc3fc None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:09.359231 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28adbe-cd3d-4258-a46d-d9e9312bc3fc None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:09.387994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:09.432530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31acf4-9bc6-4f91-a8fa-b9e0adb6e1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:09.433962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31acf4-9bc6-4f91-a8fa-b9e0adb6e1b5 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:09.434654 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31acf4-9bc6-4f91-a8fa-b9e0adb6e1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:09.435339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31acf4-9bc6-4f91-a8fa-b9e0adb6e1b5 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:09.437000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31acf4-9bc6-4f91-a8fa-b9e0adb6e1b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:09.440798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31acf4-9bc6-4f91-a8fa-b9e0adb6e1b5 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:09.569396 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 124/238] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:40:09 2022] POST /identity/v3/users => generated 293 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:40:09.612501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bf4af-5817-41a9-aebc-8b8563fb38fc None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:09.613322 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bf4af-5817-41a9-aebc-8b8563fb38fc None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:09.613322 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bf4af-5817-41a9-aebc-8b8563fb38fc None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:09.617014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 115/239] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:09.626413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740d63b-492a-4ddc-9849-441966c3bae6 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:09.627452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740d63b-492a-4ddc-9849-441966c3bae6 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:09.628312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740d63b-492a-4ddc-9849-441966c3bae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:09.671396 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-0740d63b-492a-4ddc-9849-441966c3bae6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:09.673509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-0740d63b-492a-4ddc-9849-441966c3bae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:09.714510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e3d56-64e0-4538-9e3c-75d5ec309803 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:09.714955 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e3d56-64e0-4538-9e3c-75d5ec309803 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:09.715424 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e3d56-64e0-4538-9e3c-75d5ec309803 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:09.718261 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 116/240] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:09.752865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb047246-7c63-4c62-b570-e043b37dfc1a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:09.753291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb047246-7c63-4c62-b570-e043b37dfc1a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:09.753449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb047246-7c63-4c62-b570-e043b37dfc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:09.768026 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:09.810479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-cb047246-7c63-4c62-b570-e043b37dfc1a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:09.816980 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-cb047246-7c63-4c62-b570-e043b37dfc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:09.868814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 125/241] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:09.924401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740d63b-492a-4ddc-9849-441966c3bae6 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:09.991647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 117/242] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:10.035237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb047246-7c63-4c62-b570-e043b37dfc1a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:10.128014 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:10.133854 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740d63b-492a-4ddc-9849-441966c3bae6 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:10.184419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46d94b-d1f6-4901-a6af-6e8a2458f54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:10.186041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46d94b-d1f6-4901-a6af-6e8a2458f54f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:10.186594 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46d94b-d1f6-4901-a6af-6e8a2458f54f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:10.187071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46d94b-d1f6-4901-a6af-6e8a2458f54f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:10.202357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46d94b-d1f6-4901-a6af-6e8a2458f54f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-6a46d94b-d1f6-4901-a6af-6e8a2458f54f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:10.209169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:10.210270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:10.211067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:10.211067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:10.211067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:10.211067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:10.211067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:10.211067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:10.211067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:10.211067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:10.211067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:10.211067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:10.211067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:40:10.211067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:10.214107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 126/243] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:40:10.223040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46d94b-d1f6-4901-a6af-6e8a2458f54f None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:10.290838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb047246-7c63-4c62-b570-e043b37dfc1a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:10.361016 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:10.375622 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63128375-d9e3-41e2-8c58-e59549abe2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:10.377081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128375-d9e3-41e2-8c58-e59549abe2b7 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:10.377961 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128375-d9e3-41e2-8c58-e59549abe2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:10.378790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128375-d9e3-41e2-8c58-e59549abe2b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:10.401676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63128375-d9e3-41e2-8c58-e59549abe2b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:10.404851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46d94b-d1f6-4901-a6af-6e8a2458f54f None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-63128375-d9e3-41e2-8c58-e59549abe2b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:10.416350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:10.418118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:10.419702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:10.419702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:10.419702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:10.419702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:10.419702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:10.419702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:10.419702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:10.419702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:10.419702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:10.419702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:10.419702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:10.419702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:10.424671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 118/244] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:40:10.432247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63128375-d9e3-41e2-8c58-e59549abe2b7 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:10.475677 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0933b86-8df4-4464-9d4a-e9a3a10b1142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:10.476654 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0933b86-8df4-4464-9d4a-e9a3a10b1142 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:10.477087 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0933b86-8df4-4464-9d4a-e9a3a10b1142 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:10.477564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0933b86-8df4-4464-9d4a-e9a3a10b1142 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:10.479047 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0933b86-8df4-4464-9d4a-e9a3a10b1142 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:10.479634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0933b86-8df4-4464-9d4a-e9a3a10b1142 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:10.482455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0933b86-8df4-4464-9d4a-e9a3a10b1142 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:10.506403 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 127/245] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:10.512990 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0933b86-8df4-4464-9d4a-e9a3a10b1142 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:10.561724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:10.656219 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63128375-d9e3-41e2-8c58-e59549abe2b7 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:10.713354 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0933b86-8df4-4464-9d4a-e9a3a10b1142 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:10.729842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09d8de-724b-42d6-841e-7d2864a909c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:10.731555 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09d8de-724b-42d6-841e-7d2864a909c1 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:10.732424 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09d8de-724b-42d6-841e-7d2864a909c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:10.733329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09d8de-724b-42d6-841e-7d2864a909c1 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:10.735141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09d8de-724b-42d6-841e-7d2864a909c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:10.736163 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09d8de-724b-42d6-841e-7d2864a909c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:10.739335 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09d8de-724b-42d6-841e-7d2864a909c1 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:10.757405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 119/246] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:10.769185 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09d8de-724b-42d6-841e-7d2864a909c1 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:10.782290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07cb74-cb3f-49ed-8ae5-e937c6a14261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:10.783703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07cb74-cb3f-49ed-8ae5-e937c6a14261 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:10.784476 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07cb74-cb3f-49ed-8ae5-e937c6a14261 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:10.785089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07cb74-cb3f-49ed-8ae5-e937c6a14261 None admin] PATH_INFO: `/v3/users/248db7da4a88429fb74819edb303200f` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:10.812986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07cb74-cb3f-49ed-8ae5-e937c6a14261 None admin] RBAC: Authorizing `identity:get_user(user_id=248db7da4a88429fb74819edb303200f)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:10.815778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07cb74-cb3f-49ed-8ae5-e937c6a14261 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:10.819726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 128/247] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:10 2022] GET /identity/v3/users/248db7da4a88429fb74819edb303200f => generated 297 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:10.826464 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07cb74-cb3f-49ed-8ae5-e937c6a14261 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:10.947416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:10.984974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:11.039654 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07cb74-cb3f-49ed-8ae5-e937c6a14261 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:11.102083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09d8de-724b-42d6-841e-7d2864a909c1 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:11.112938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da733e4-5428-4c89-ab03-e635da5d80b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:11.115568 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da733e4-5428-4c89-ab03-e635da5d80b4 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:11.116481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da733e4-5428-4c89-ab03-e635da5d80b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:11.117176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da733e4-5428-4c89-ab03-e635da5d80b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:11.222276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcc68c-eca2-4ae5-88d8-5022fcb45e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:11.222850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcc68c-eca2-4ae5-88d8-5022fcb45e94 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:11.223138 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcc68c-eca2-4ae5-88d8-5022fcb45e94 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:11.224267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcc68c-eca2-4ae5-88d8-5022fcb45e94 None admin] PATH_INFO: `/v3/users/4025368a8cf44d6ebecb90392d7b799a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:11.228586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcc68c-eca2-4ae5-88d8-5022fcb45e94 None admin] RBAC: Authorizing `identity:get_user(user_id=4025368a8cf44d6ebecb90392d7b799a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:11.230059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fcc68c-eca2-4ae5-88d8-5022fcb45e94 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:11.232331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 120/248] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:10 2022] GET /identity/v3/users/4025368a8cf44d6ebecb90392d7b799a => generated 297 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:11.239952 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcc68c-eca2-4ae5-88d8-5022fcb45e94 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:11.297362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da733e4-5428-4c89-ab03-e635da5d80b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:11.301366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da733e4-5428-4c89-ab03-e635da5d80b4 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:11.446490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fcc68c-eca2-4ae5-88d8-5022fcb45e94 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:11.476232 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:11.480334 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:11.508346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab173cd-65e6-4eac-95c5-769757ab0c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:11.509090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab173cd-65e6-4eac-95c5-769757ab0c5d None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:11.509381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab173cd-65e6-4eac-95c5-769757ab0c5d None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:11.509668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab173cd-65e6-4eac-95c5-769757ab0c5d None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:11.511582 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab173cd-65e6-4eac-95c5-769757ab0c5d None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:11.512841 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab173cd-65e6-4eac-95c5-769757ab0c5d None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:11.521140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 121/249] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:11 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:11.528605 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16599b1e-39c4-4cb3-bb5b-f5cef3faf272 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:11.528605 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16599b1e-39c4-4cb3-bb5b-f5cef3faf272 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:11.528605 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16599b1e-39c4-4cb3-bb5b-f5cef3faf272 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:11.528605 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 122/250] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:11.528605 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16599b1e-39c4-4cb3-bb5b-f5cef3faf272 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:11.534888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 129/251] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 709 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:11.538187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462e111-d896-4d17-bdb3-4c03e2caa958 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:11.539084 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462e111-d896-4d17-bdb3-4c03e2caa958 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:11.539948 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462e111-d896-4d17-bdb3-4c03e2caa958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:11.575918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-4462e111-d896-4d17-bdb3-4c03e2caa958 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:11.578880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4462e111-d896-4d17-bdb3-4c03e2caa958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:11.730508 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 130/252] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:11.732614 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462e111-d896-4d17-bdb3-4c03e2caa958 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:11.875728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16599b1e-39c4-4cb3-bb5b-f5cef3faf272 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:11.922531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43880025-80a7-4961-8aa8-49079613c691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:11.922531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43880025-80a7-4961-8aa8-49079613c691 None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:11.922531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43880025-80a7-4961-8aa8-49079613c691 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:11.922531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43880025-80a7-4961-8aa8-49079613c691 None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0/users/4025368a8cf44d6ebecb90392d7b799a/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:11.923134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43880025-80a7-4961-8aa8-49079613c691 None admin] RBAC: Authorizing `identity:create_grant(project_id=21ad3a8335ad492d858142d787614be0, user_id=4025368a8cf44d6ebecb90392d7b799a, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:11.924469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43880025-80a7-4961-8aa8-49079613c691 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:11.962499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462e111-d896-4d17-bdb3-4c03e2caa958 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:11.981935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:12.032778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 123/253] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:40:11 2022] PUT /identity/v3/projects/21ad3a8335ad492d858142d787614be0/users/4025368a8cf44d6ebecb90392d7b799a/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:40:12.033880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43880025-80a7-4961-8aa8-49079613c691 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:12.044428 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff1385-2242-4883-a9a9-c66ce4810378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:12.047103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff1385-2242-4883-a9a9-c66ce4810378 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:12.048402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff1385-2242-4883-a9a9-c66ce4810378 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:12.049138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff1385-2242-4883-a9a9-c66ce4810378 None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:12.056855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff1385-2242-4883-a9a9-c66ce4810378 None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:12.064484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff1385-2242-4883-a9a9-c66ce4810378 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:12.073260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 131/254] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:11 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:12.079021 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff1385-2242-4883-a9a9-c66ce4810378 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:12.253629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:12.407819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff1385-2242-4883-a9a9-c66ce4810378 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:12.451384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43880025-80a7-4961-8aa8-49079613c691 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:12.484100 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ad62f-e97d-4807-a8e1-e1e50e526f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:12.485778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ad62f-e97d-4807-a8e1-e1e50e526f5c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:12.486551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ad62f-e97d-4807-a8e1-e1e50e526f5c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:12.487391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ad62f-e97d-4807-a8e1-e1e50e526f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:12.490418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ad62f-e97d-4807-a8e1-e1e50e526f5c None admin] RBAC: Adding query filter params (role.id=cf7c52610e954f74a30feba325390387, scope.project.id=21ad3a8335ad492d858142d787614be0, user.id=248db7da4a88429fb74819edb303200f) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:12.491543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ad62f-e97d-4807-a8e1-e1e50e526f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:12.494800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ad62f-e97d-4807-a8e1-e1e50e526f5c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:12.501313 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f22d8a-3243-462e-8436-6d71f142b0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:12.501949 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f22d8a-3243-462e-8436-6d71f142b0f9 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:12.502233 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f22d8a-3243-462e-8436-6d71f142b0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:12.502597 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f22d8a-3243-462e-8436-6d71f142b0f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:12.508382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 132/255] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:40:12 2022] GET /identity/v3/role_assignments?user.id=248db7da4a88429fb74819edb303200f&scope.project.id=21ad3a8335ad492d858142d787614be0&role.id=cf7c52610e954f74a30feba325390387 => generated 259 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:12.511026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9691173-61c4-4886-b9fd-30b68d48b7a2 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:12.511918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9691173-61c4-4886-b9fd-30b68d48b7a2 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:12.512765 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9691173-61c4-4886-b9fd-30b68d48b7a2 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:12.515846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 133/256] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:12.529416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00025880-52c3-4f19-8611-154f1399561c None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:12.530388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00025880-52c3-4f19-8611-154f1399561c None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:12.531168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00025880-52c3-4f19-8611-154f1399561c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:12.539278 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f22d8a-3243-462e-8436-6d71f142b0f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-c9f22d8a-3243-462e-8436-6d71f142b0f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:12.546017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:12.547908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:12.549508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:12.549508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:12.549508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:12.549508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:12.549508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:12.549508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:12.549508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:12.549508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:12.549508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:12.549508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:12.549508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:40:12.549508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:12.549508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 124/257] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:40:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 515 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:12.559043 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f22d8a-3243-462e-8436-6d71f142b0f9 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:12.586723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-00025880-52c3-4f19-8611-154f1399561c None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:12.588812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-00025880-52c3-4f19-8611-154f1399561c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:12.751154 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:12.877790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f22d8a-3243-462e-8436-6d71f142b0f9 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:12.885217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 134/258] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:12.904096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:12.932126 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00025880-52c3-4f19-8611-154f1399561c None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:12.954549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f83fb6-f98f-4f04-bf3a-0683dae3ab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:12.955662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f83fb6-f98f-4f04-bf3a-0683dae3ab9b None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:12.956231 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f83fb6-f98f-4f04-bf3a-0683dae3ab9b None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:12.956653 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f83fb6-f98f-4f04-bf3a-0683dae3ab9b None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:12.958024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f83fb6-f98f-4f04-bf3a-0683dae3ab9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:12.958584 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f83fb6-f98f-4f04-bf3a-0683dae3ab9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:12.961476 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f83fb6-f98f-4f04-bf3a-0683dae3ab9b None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:12.978591 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 125/259] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:12.988296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f83fb6-f98f-4f04-bf3a-0683dae3ab9b None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:13.253969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f83fb6-f98f-4f04-bf3a-0683dae3ab9b None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:13.260582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00025880-52c3-4f19-8611-154f1399561c None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:13.336592 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a38c7-b8f0-4bce-a4e4-0d7c25308516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:13.337292 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a38c7-b8f0-4bce-a4e4-0d7c25308516 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:13.337578 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a38c7-b8f0-4bce-a4e4-0d7c25308516 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:13.337862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a38c7-b8f0-4bce-a4e4-0d7c25308516 None admin] PATH_INFO: `/v3/users/d63c97bc953b43dc83b6633361d498df` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:13.341411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b000b-93f9-4aa9-84f9-035f81be0a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:13.342081 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b000b-93f9-4aa9-84f9-035f81be0a6b None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:13.342363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b000b-93f9-4aa9-84f9-035f81be0a6b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:13.342627 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b000b-93f9-4aa9-84f9-035f81be0a6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:13.344107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a38c7-b8f0-4bce-a4e4-0d7c25308516 None admin] RBAC: Authorizing `identity:get_user(user_id=d63c97bc953b43dc83b6633361d498df)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:13.345655 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a38c7-b8f0-4bce-a4e4-0d7c25308516 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:13.347436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 126/260] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:12 2022] GET /identity/v3/users/d63c97bc953b43dc83b6633361d498df => generated 283 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:13.360747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a38c7-b8f0-4bce-a4e4-0d7c25308516 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:13.388108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b000b-93f9-4aa9-84f9-035f81be0a6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-973b000b-93f9-4aa9-84f9-035f81be0a6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:13.400031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:13.401742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:13.403062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:13.403062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:13.403062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:13.403062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:13.403062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:13.403062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:13.403062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:13.403062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:13.403062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:13.403062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:13.403062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:13.403062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:13.403062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 135/261] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:40:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 470 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:40:13.416676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b000b-93f9-4aa9-84f9-035f81be0a6b None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:13.507600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:13.522815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:13.692008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b000b-93f9-4aa9-84f9-035f81be0a6b None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:13.741614 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a38c7-b8f0-4bce-a4e4-0d7c25308516 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:13.792366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283cf80-0aab-4984-afa6-e73a27ebe9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:13.793127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283cf80-0aab-4984-afa6-e73a27ebe9f8 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:13.793415 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283cf80-0aab-4984-afa6-e73a27ebe9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:13.793696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283cf80-0aab-4984-afa6-e73a27ebe9f8 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:13.794542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283cf80-0aab-4984-afa6-e73a27ebe9f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:13.795087 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283cf80-0aab-4984-afa6-e73a27ebe9f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:13.796755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283cf80-0aab-4984-afa6-e73a27ebe9f8 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:13.815525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 136/262] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:40:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:13.819204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283cf80-0aab-4984-afa6-e73a27ebe9f8 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:13.854624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b7184-8266-4150-a9e3-7cd97f667892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:13.855361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b7184-8266-4150-a9e3-7cd97f667892 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:13.855678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b7184-8266-4150-a9e3-7cd97f667892 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:13.855959 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b7184-8266-4150-a9e3-7cd97f667892 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:14.048336 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b7184-8266-4150-a9e3-7cd97f667892 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:14.049893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b7184-8266-4150-a9e3-7cd97f667892 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:14.056864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:14.079924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283cf80-0aab-4984-afa6-e73a27ebe9f8 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:14.157515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331a49b-974c-4e51-82e2-dc365a664e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:14.158391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331a49b-974c-4e51-82e2-dc365a664e8a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:14.158678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331a49b-974c-4e51-82e2-dc365a664e8a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:14.158959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331a49b-974c-4e51-82e2-dc365a664e8a None admin] PATH_INFO: `/v3/users/d63c97bc953b43dc83b6633361d498df` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:14.164534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331a49b-974c-4e51-82e2-dc365a664e8a None admin] RBAC: Authorizing `identity:get_user(user_id=d63c97bc953b43dc83b6633361d498df)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:14.166070 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331a49b-974c-4e51-82e2-dc365a664e8a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:14.171339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 137/263] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:13 2022] GET /identity/v3/users/d63c97bc953b43dc83b6633361d498df => generated 283 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:14.174530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331a49b-974c-4e51-82e2-dc365a664e8a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:14.194031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:14.450838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 127/264] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1090 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:14.456827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b7184-8266-4150-a9e3-7cd97f667892 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:14.530594 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331a49b-974c-4e51-82e2-dc365a664e8a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:14.668198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe6536-e903-4a7b-92d5-0095504c6129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:14.669813 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe6536-e903-4a7b-92d5-0095504c6129 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:14.670785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe6536-e903-4a7b-92d5-0095504c6129 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:14.671719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe6536-e903-4a7b-92d5-0095504c6129 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:14.684308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b7184-8266-4150-a9e3-7cd97f667892 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:14.747735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:14.779086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0abed-0995-4453-9bc4-496721d5e85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:14.780231 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0abed-0995-4453-9bc4-496721d5e85f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:14.780678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0abed-0995-4453-9bc4-496721d5e85f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:14.781126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0abed-0995-4453-9bc4-496721d5e85f None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:14.788797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0abed-0995-4453-9bc4-496721d5e85f None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:14.791371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0abed-0995-4453-9bc4-496721d5e85f None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:14.794672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 128/265] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:14 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:14.798338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe6536-e903-4a7b-92d5-0095504c6129 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:14.804567 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe6536-e903-4a7b-92d5-0095504c6129 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:14.821177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0abed-0995-4453-9bc4-496721d5e85f None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:14.856598 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:14.964790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 138/266] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 791 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:14.977630 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daea4a2-940c-4122-ae1e-42a63e3f1754 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:14.978577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daea4a2-940c-4122-ae1e-42a63e3f1754 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:14.979312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daea4a2-940c-4122-ae1e-42a63e3f1754 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:14.982538 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 139/267] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:14.984927 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daea4a2-940c-4122-ae1e-42a63e3f1754 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:15.144191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0abed-0995-4453-9bc4-496721d5e85f None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:15.188897 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:15.200100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99874fc-e227-495f-8e22-da111d8a7556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:15.212777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99874fc-e227-495f-8e22-da111d8a7556 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:15.212777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99874fc-e227-495f-8e22-da111d8a7556 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:15.212777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99874fc-e227-495f-8e22-da111d8a7556 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:15.212777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99874fc-e227-495f-8e22-da111d8a7556 None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=21ad3a8335ad492d858142d787614be0, user.id=d63c97bc953b43dc83b6633361d498df) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:15.212777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99874fc-e227-495f-8e22-da111d8a7556 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:15.212777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99874fc-e227-495f-8e22-da111d8a7556 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:15.225736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 129/268] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:40:14 2022] GET /identity/v3/role_assignments?user.id=d63c97bc953b43dc83b6633361d498df&scope.project.id=21ad3a8335ad492d858142d787614be0&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 608 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:15.228519 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c353e4-eb4d-47d1-a918-de1c20f06760 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:15.229389 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c353e4-eb4d-47d1-a918-de1c20f06760 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:15.230237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c353e4-eb4d-47d1-a918-de1c20f06760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:15.275146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daea4a2-940c-4122-ae1e-42a63e3f1754 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:15.294645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-c5c353e4-eb4d-47d1-a918-de1c20f06760 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:15.297363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c5c353e4-eb4d-47d1-a918-de1c20f06760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:15.349826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db01a31-a850-4adc-9144-20a0e658690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:15.350952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db01a31-a850-4adc-9144-20a0e658690e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:15.351437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db01a31-a850-4adc-9144-20a0e658690e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:15.351971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db01a31-a850-4adc-9144-20a0e658690e None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:15.354360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db01a31-a850-4adc-9144-20a0e658690e None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:15.364997 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db01a31-a850-4adc-9144-20a0e658690e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:15.367877 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 140/269] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:14 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:15.369382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dce947-4d45-4969-8436-288e44375ea1 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:15.369693 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dce947-4d45-4969-8436-288e44375ea1 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:15.369981 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dce947-4d45-4969-8436-288e44375ea1 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:15.371553 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 141/270] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:15.379138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6444c3-9124-41ab-9041-ebee93003970 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:15.379456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6444c3-9124-41ab-9041-ebee93003970 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:15.379824 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6444c3-9124-41ab-9041-ebee93003970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:15.419665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-fe6444c3-9124-41ab-9041-ebee93003970 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:15.419665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-fe6444c3-9124-41ab-9041-ebee93003970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:15.481821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 130/271] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:15.483860 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c353e4-eb4d-47d1-a918-de1c20f06760 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:15.489052 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:15.539580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 142/272] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:40:15.557988 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6444c3-9124-41ab-9041-ebee93003970 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:15.710336 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c353e4-eb4d-47d1-a918-de1c20f06760 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:15.750597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:15.779523 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e32a91-5402-40c0-82f0-3d2602ec0b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:15.780526 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e32a91-5402-40c0-82f0-3d2602ec0b72 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:15.780904 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e32a91-5402-40c0-82f0-3d2602ec0b72 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:15.781322 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e32a91-5402-40c0-82f0-3d2602ec0b72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:15.782801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e32a91-5402-40c0-82f0-3d2602ec0b72 None admin] RBAC: Adding query filter params (role.id=eff5cca513be4ed5a9b001c0559fb476, scope.project.id=21ad3a8335ad492d858142d787614be0, user.id=d63c97bc953b43dc83b6633361d498df) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:15.783392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e32a91-5402-40c0-82f0-3d2602ec0b72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:15.786827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e32a91-5402-40c0-82f0-3d2602ec0b72 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:15.798178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 131/273] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:40:15 2022] GET /identity/v3/role_assignments?user.id=d63c97bc953b43dc83b6633361d498df&scope.project.id=21ad3a8335ad492d858142d787614be0&role.id=eff5cca513be4ed5a9b001c0559fb476 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:15.801801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e32a91-5402-40c0-82f0-3d2602ec0b72 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:15.867826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6444c3-9124-41ab-9041-ebee93003970 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:15.944578 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:15.963614 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec6de6-418a-4083-864c-dcbec9e120be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:15.965198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec6de6-418a-4083-864c-dcbec9e120be None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:15.965962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec6de6-418a-4083-864c-dcbec9e120be None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:15.966700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec6de6-418a-4083-864c-dcbec9e120be None admin] PATH_INFO: `/v3/roles/member` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:15.992668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec6de6-418a-4083-864c-dcbec9e120be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-58ec6de6-418a-4083-864c-dcbec9e120be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:16.002994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:16.005587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:16.007200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:16.007200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:16.007200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:16.007200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:16.007200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:16.007200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:16.007200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:16.007200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:16.007200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:16.007200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:16.007200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:16.007200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:16.017643 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e32a91-5402-40c0-82f0-3d2602ec0b72 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:16.018276 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 143/274] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 455 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:40:16.021601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0a6da-b88d-40ee-990f-8fb9d411d50b None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:16.022479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0a6da-b88d-40ee-990f-8fb9d411d50b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:16.023266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0a6da-b88d-40ee-990f-8fb9d411d50b None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:16.028422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 144/275] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:16.040272 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff4dda-6bc2-4bf9-8135-41785f52bf37 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:16.041317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff4dda-6bc2-4bf9-8135-41785f52bf37 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:16.042038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff4dda-6bc2-4bf9-8135-41785f52bf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:16.076393 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c4ff4dda-6bc2-4bf9-8135-41785f52bf37 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:16.078958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c4ff4dda-6bc2-4bf9-8135-41785f52bf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:16.090550 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d498fc0-b785-4651-bc76-b91df0c094a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:16.091732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d498fc0-b785-4651-bc76-b91df0c094a4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:16.092208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d498fc0-b785-4651-bc76-b91df0c094a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:16.092757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d498fc0-b785-4651-bc76-b91df0c094a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:16.095034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d498fc0-b785-4651-bc76-b91df0c094a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:16.097742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d498fc0-b785-4651-bc76-b91df0c094a4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:16.101349 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 132/276] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:40:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:16.102850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d498fc0-b785-4651-bc76-b91df0c094a4 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:16.216292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 145/277] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:40:16.217753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff4dda-6bc2-4bf9-8135-41785f52bf37 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:16.287312 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d498fc0-b785-4651-bc76-b91df0c094a4 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:16.292899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:16.338428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b5827-0d2c-4f5e-8faa-0152a7c3b4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:16.339187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b5827-0d2c-4f5e-8faa-0152a7c3b4dd None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:16.339543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b5827-0d2c-4f5e-8faa-0152a7c3b4dd None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:16.339791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b5827-0d2c-4f5e-8faa-0152a7c3b4dd None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:16.340553 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b5827-0d2c-4f5e-8faa-0152a7c3b4dd None admin] RBAC: Adding query filter params (name=member) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:16.340984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b5827-0d2c-4f5e-8faa-0152a7c3b4dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:16.342513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b5827-0d2c-4f5e-8faa-0152a7c3b4dd None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:16.344559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:16.354538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 133/278] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:16.359499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff4dda-6bc2-4bf9-8135-41785f52bf37 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:16.360370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b5827-0d2c-4f5e-8faa-0152a7c3b4dd None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:16.404689 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357a2b6-5674-46d2-a4ec-1b4f9678ca9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:16.405329 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357a2b6-5674-46d2-a4ec-1b4f9678ca9e None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:16.405614 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357a2b6-5674-46d2-a4ec-1b4f9678ca9e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:16.405870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e357a2b6-5674-46d2-a4ec-1b4f9678ca9e None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:16.406650 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357a2b6-5674-46d2-a4ec-1b4f9678ca9e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:16.408077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e357a2b6-5674-46d2-a4ec-1b4f9678ca9e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:16.493046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b5827-0d2c-4f5e-8faa-0152a7c3b4dd None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:16.512541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 146/279] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:40:16 2022] POST /identity/v3/users => generated 293 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:40:16.514195 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357a2b6-5674-46d2-a4ec-1b4f9678ca9e None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:16.560814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc8905-9c5b-4a90-9df1-66772bdd5a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:16.561901 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc8905-9c5b-4a90-9df1-66772bdd5a3f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:16.562359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc8905-9c5b-4a90-9df1-66772bdd5a3f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:16.562790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc8905-9c5b-4a90-9df1-66772bdd5a3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:16.585895 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc8905-9c5b-4a90-9df1-66772bdd5a3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-d6dc8905-9c5b-4a90-9df1-66772bdd5a3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:16.595191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:16.596384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:16.597272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:16.597272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:16.597272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:16.597272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:16.597272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:16.597272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:16.597272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:16.597272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:16.597272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:16.597272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:16.597272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:16.597272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:16.604337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 134/280] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:16.608513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc8905-9c5b-4a90-9df1-66772bdd5a3f None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:16.672130 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e357a2b6-5674-46d2-a4ec-1b4f9678ca9e None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:16.678132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:16.745788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8c22a-37ca-4702-83b4-89540ebffb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:16.747522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8c22a-37ca-4702-83b4-89540ebffb5a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:16.748890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8c22a-37ca-4702-83b4-89540ebffb5a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:16.749894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8c22a-37ca-4702-83b4-89540ebffb5a None admin] PATH_INFO: `/v3/users/ae9e318c3bf7440f8b645235a8855ba6` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:16.756215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:16.780591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8c22a-37ca-4702-83b4-89540ebffb5a None admin] RBAC: Authorizing `identity:get_user(user_id=ae9e318c3bf7440f8b645235a8855ba6)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:16.784272 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8c22a-37ca-4702-83b4-89540ebffb5a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:16.789057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 147/281] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:16 2022] GET /identity/v3/users/ae9e318c3bf7440f8b645235a8855ba6 => generated 293 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:16.801783 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8c22a-37ca-4702-83b4-89540ebffb5a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:16.862332 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc8905-9c5b-4a90-9df1-66772bdd5a3f None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:16.930405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504e366-f8a1-4d85-8082-1caab8db3c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:16.931480 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504e366-f8a1-4d85-8082-1caab8db3c2b None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:16.931959 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504e366-f8a1-4d85-8082-1caab8db3c2b None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:16.933428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504e366-f8a1-4d85-8082-1caab8db3c2b None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:16.934580 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504e366-f8a1-4d85-8082-1caab8db3c2b None admin] RBAC: Adding query filter params (name=member) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:16.935062 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504e366-f8a1-4d85-8082-1caab8db3c2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:16.937574 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504e366-f8a1-4d85-8082-1caab8db3c2b None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:16.953600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 135/282] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:16.955652 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9ac09-0400-44f1-997f-931d43af6562 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:16.956069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9ac09-0400-44f1-997f-931d43af6562 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:16.956481 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9ac09-0400-44f1-997f-931d43af6562 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:16.959570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 136/283] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:16.968620 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa0fac-8bc9-4883-80c7-323010bb2435 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:16.969028 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa0fac-8bc9-4883-80c7-323010bb2435 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:16.969449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa0fac-8bc9-4883-80c7-323010bb2435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:16.999407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-06fa0fac-8bc9-4883-80c7-323010bb2435 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:17.000740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-06fa0fac-8bc9-4883-80c7-323010bb2435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:17.004803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8c22a-37ca-4702-83b4-89540ebffb5a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:17.076956 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb752db6-0488-4e37-b9c0-3c91f0ec3821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:17.078482 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb752db6-0488-4e37-b9c0-3c91f0ec3821 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:17.079403 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb752db6-0488-4e37-b9c0-3c91f0ec3821 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:17.080186 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb752db6-0488-4e37-b9c0-3c91f0ec3821 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:17.082046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb752db6-0488-4e37-b9c0-3c91f0ec3821 None admin] RBAC: Adding query filter params (role.id=6a924f1d619643b3be92bab6fddb3eb2, user.id=ae9e318c3bf7440f8b645235a8855ba6, scope.system=all) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:17.083482 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb752db6-0488-4e37-b9c0-3c91f0ec3821 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:17.086590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb752db6-0488-4e37-b9c0-3c91f0ec3821 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:17.096253 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 148/284] 10.222.0.94 () {56 vars in 1308 bytes} [Tue Jul 26 14:40:16 2022] GET /identity/v3/role_assignments?user.id=ae9e318c3bf7440f8b645235a8855ba6&scope.system=all&role.id=6a924f1d619643b3be92bab6fddb3eb2 => generated 226 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:17.097606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb752db6-0488-4e37-b9c0-3c91f0ec3821 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:17.136231 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 137/285] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:17.139851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:17.172789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa0fac-8bc9-4883-80c7-323010bb2435 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:17.241001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb752db6-0488-4e37-b9c0-3c91f0ec3821 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:17.276653 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:17.310062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c828edc-1ca9-412c-82c6-9f5823324feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:17.311529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c828edc-1ca9-412c-82c6-9f5823324feb None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:17.312072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c828edc-1ca9-412c-82c6-9f5823324feb None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:17.312546 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c828edc-1ca9-412c-82c6-9f5823324feb None admin] PATH_INFO: `/v3/users/4025368a8cf44d6ebecb90392d7b799a` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:17.314206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c828edc-1ca9-412c-82c6-9f5823324feb None admin] RBAC: Authorizing `identity:get_user(user_id=4025368a8cf44d6ebecb90392d7b799a)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:17.316126 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c828edc-1ca9-412c-82c6-9f5823324feb None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:17.318810 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 149/286] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:17 2022] GET /identity/v3/users/4025368a8cf44d6ebecb90392d7b799a => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:17.370916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c828edc-1ca9-412c-82c6-9f5823324feb None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:17.389767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa0fac-8bc9-4883-80c7-323010bb2435 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:17.466738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9428600-b264-43d3-8a0a-5f112c79d196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:17.467887 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9428600-b264-43d3-8a0a-5f112c79d196 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:17.468675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9428600-b264-43d3-8a0a-5f112c79d196 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:17.468882 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9428600-b264-43d3-8a0a-5f112c79d196 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:17.502187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9428600-b264-43d3-8a0a-5f112c79d196 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-c9428600-b264-43d3-8a0a-5f112c79d196 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:17.520500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:17.522321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:17.524742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:17.524742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:17.524742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:17.524742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:17.524742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:17.524742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:17.524742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:17.524742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:17.524742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:17.524742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:17.524742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:40:17.524742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:17.524742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 138/287] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:17.536113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9428600-b264-43d3-8a0a-5f112c79d196 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:17.580155 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:17.607558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c828edc-1ca9-412c-82c6-9f5823324feb None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:17.689939 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9428600-b264-43d3-8a0a-5f112c79d196 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:17.711528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:17.724182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d8b0e-f313-4e7c-b492-1c81ca2c848e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:17.726039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d8b0e-f313-4e7c-b492-1c81ca2c848e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:17.726188 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d8b0e-f313-4e7c-b492-1c81ca2c848e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:17.726594 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d8b0e-f313-4e7c-b492-1c81ca2c848e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:17.811104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525656d8-53bb-4af8-b120-eae3b258c91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:17.812137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525656d8-53bb-4af8-b120-eae3b258c91b None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:17.812621 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525656d8-53bb-4af8-b120-eae3b258c91b None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:17.813120 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525656d8-53bb-4af8-b120-eae3b258c91b None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:17.815168 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525656d8-53bb-4af8-b120-eae3b258c91b None admin] RBAC: Adding query filter params (name=reader) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:17.815640 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525656d8-53bb-4af8-b120-eae3b258c91b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:17.820200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525656d8-53bb-4af8-b120-eae3b258c91b None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:17.836824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 139/288] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:17.845333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525656d8-53bb-4af8-b120-eae3b258c91b None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:17.935951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d8b0e-f313-4e7c-b492-1c81ca2c848e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:17.938714 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d8b0e-f313-4e7c-b492-1c81ca2c848e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:18.054770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525656d8-53bb-4af8-b120-eae3b258c91b None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:18.112663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989390b-5766-419e-97ee-31098e340d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:18.113277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989390b-5766-419e-97ee-31098e340d13 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:18.113562 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989390b-5766-419e-97ee-31098e340d13 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:18.114005 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989390b-5766-419e-97ee-31098e340d13 None admin] PATH_INFO: `/v3/users/248db7da4a88429fb74819edb303200f` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:18.116201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989390b-5766-419e-97ee-31098e340d13 None admin] RBAC: Authorizing `identity:get_user(user_id=248db7da4a88429fb74819edb303200f)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:18.117803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989390b-5766-419e-97ee-31098e340d13 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:18.122771 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 140/289] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:17 2022] GET /identity/v3/users/248db7da4a88429fb74819edb303200f => generated 297 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:18.140354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989390b-5766-419e-97ee-31098e340d13 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:18.170094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:18.184216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 150/290] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 857 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:18.197114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d8b0e-f313-4e7c-b492-1c81ca2c848e None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:18.229082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:18.356148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989390b-5766-419e-97ee-31098e340d13 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:18.423240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04861570-e4c4-432f-a7d3-85f854a86f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:18.424060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04861570-e4c4-432f-a7d3-85f854a86f19 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:18.424334 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04861570-e4c4-432f-a7d3-85f854a86f19 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:18.424644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04861570-e4c4-432f-a7d3-85f854a86f19 None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:18.432501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04861570-e4c4-432f-a7d3-85f854a86f19 None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:18.434072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04861570-e4c4-432f-a7d3-85f854a86f19 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:18.435979 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 141/291] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:18 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:18.458815 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04861570-e4c4-432f-a7d3-85f854a86f19 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:18.521655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d8b0e-f313-4e7c-b492-1c81ca2c848e None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:18.633589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38fea3-106a-4cbc-895a-188178bad845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:18.634772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38fea3-106a-4cbc-895a-188178bad845 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:18.635285 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38fea3-106a-4cbc-895a-188178bad845 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:18.635788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38fea3-106a-4cbc-895a-188178bad845 None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:18.638964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c38fea3-106a-4cbc-895a-188178bad845 None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:18.640942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c38fea3-106a-4cbc-895a-188178bad845 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:18.643049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 151/292] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:18 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:18.654078 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38fea3-106a-4cbc-895a-188178bad845 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:18.696202 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04861570-e4c4-432f-a7d3-85f854a86f19 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:18.708552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:18.783551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:18.804321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3149064-1614-4f4e-ab29-e82abe8b6ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:18.805043 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3149064-1614-4f4e-ab29-e82abe8b6ad3 None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:18.805334 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3149064-1614-4f4e-ab29-e82abe8b6ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:18.805611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3149064-1614-4f4e-ab29-e82abe8b6ad3 None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0/users/248db7da4a88429fb74819edb303200f/roles/cf7c52610e954f74a30feba325390387` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:18.809080 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3149064-1614-4f4e-ab29-e82abe8b6ad3 None admin] RBAC: Authorizing `identity:create_grant(project_id=21ad3a8335ad492d858142d787614be0, user_id=248db7da4a88429fb74819edb303200f, role_id=cf7c52610e954f74a30feba325390387)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:18.810603 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3149064-1614-4f4e-ab29-e82abe8b6ad3 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:18.861644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38fea3-106a-4cbc-895a-188178bad845 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:18.888439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 142/293] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:40:18 2022] PUT /identity/v3/projects/21ad3a8335ad492d858142d787614be0/users/248db7da4a88429fb74819edb303200f/roles/cf7c52610e954f74a30feba325390387 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:40:18.921042 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf918e1-ddbe-474a-826e-a999d8fe5e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:18.921673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf918e1-ddbe-474a-826e-a999d8fe5e2d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:18.921953 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf918e1-ddbe-474a-826e-a999d8fe5e2d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:18.922226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf918e1-ddbe-474a-826e-a999d8fe5e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:18.923036 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf918e1-ddbe-474a-826e-a999d8fe5e2d None admin] RBAC: Adding query filter params (role.id=6a924f1d619643b3be92bab6fddb3eb2, scope.project.id=21ad3a8335ad492d858142d787614be0, user.id=4025368a8cf44d6ebecb90392d7b799a) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:18.923350 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf918e1-ddbe-474a-826e-a999d8fe5e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:18.925029 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf918e1-ddbe-474a-826e-a999d8fe5e2d None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:18.933512 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 152/294] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:40:18 2022] GET /identity/v3/role_assignments?user.id=4025368a8cf44d6ebecb90392d7b799a&scope.project.id=21ad3a8335ad492d858142d787614be0&role.id=6a924f1d619643b3be92bab6fddb3eb2 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:19.852010 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063b413-77ab-41ba-89a7-9de3df79c1b1 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:19.852650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063b413-77ab-41ba-89a7-9de3df79c1b1 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:19.852650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063b413-77ab-41ba-89a7-9de3df79c1b1 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:19.854487 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 143/295] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:19.860958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f50f5-f5cb-4c85-bf7b-612fd2a3a649 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:19.861517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f50f5-f5cb-4c85-bf7b-612fd2a3a649 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:19.861517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f50f5-f5cb-4c85-bf7b-612fd2a3a649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:19.883834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-f80f50f5-f5cb-4c85-bf7b-612fd2a3a649 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:19.885282 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f80f50f5-f5cb-4c85-bf7b-612fd2a3a649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:19.973168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 153/296] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:20.012931 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063b413-77ab-41ba-89a7-9de3df79c1b1 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:20.167881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1f060-723c-46ec-97b9-b3824bdc5c47 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:20.168809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1f060-723c-46ec-97b9-b3824bdc5c47 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:20.169452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1f060-723c-46ec-97b9-b3824bdc5c47 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:20.171439 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 154/297] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:20.178179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369532f-1539-4ae0-a00e-1ce20a526648 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:20.178880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369532f-1539-4ae0-a00e-1ce20a526648 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:20.179432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369532f-1539-4ae0-a00e-1ce20a526648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:20.184278 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063b413-77ab-41ba-89a7-9de3df79c1b1 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:20.201914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c369532f-1539-4ae0-a00e-1ce20a526648 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:20.204531 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c369532f-1539-4ae0-a00e-1ce20a526648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:20.250800 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa928ed-a925-45e9-badb-32555e4f4b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:20.251758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa928ed-a925-45e9-badb-32555e4f4b98 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:20.252236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa928ed-a925-45e9-badb-32555e4f4b98 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:20.254416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa928ed-a925-45e9-badb-32555e4f4b98 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:20.276469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa928ed-a925-45e9-badb-32555e4f4b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-caa928ed-a925-45e9-badb-32555e4f4b98 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:20.284069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:20.285160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:20.286530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:20.286530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:20.286530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:20.286530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:20.286530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:20.286530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:20.286530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:20.286530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:20.286530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:20.286530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:20.286530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:40:20.286530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:20.289745 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 144/298] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:20.299595 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa928ed-a925-45e9-badb-32555e4f4b98 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:20.311529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:20.343554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 155/299] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:20.364187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:20.371318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40741845-95cd-441d-8f96-ae0cb905885b None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:20.371620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40741845-95cd-441d-8f96-ae0cb905885b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:20.371904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40741845-95cd-441d-8f96-ae0cb905885b None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:20.373469 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 156/300] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:40:20.379059 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40741845-95cd-441d-8f96-ae0cb905885b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:20.463902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa928ed-a925-45e9-badb-32555e4f4b98 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:20.507210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b65d4f-0463-4978-81ac-b359f5105673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:20.507877 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b65d4f-0463-4978-81ac-b359f5105673 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:20.508161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b65d4f-0463-4978-81ac-b359f5105673 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:20.508439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b65d4f-0463-4978-81ac-b359f5105673 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:20.509249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b65d4f-0463-4978-81ac-b359f5105673 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:20.509574 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b65d4f-0463-4978-81ac-b359f5105673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:20.510998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b65d4f-0463-4978-81ac-b359f5105673 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:20.520784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 145/301] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:20.522140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae5801-6e30-4fa0-9a95-6ebbd7622ca6 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:20.522507 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae5801-6e30-4fa0-9a95-6ebbd7622ca6 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:20.522804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae5801-6e30-4fa0-9a95-6ebbd7622ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:20.544644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-e2ae5801-6e30-4fa0-9a95-6ebbd7622ca6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:20.545946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e2ae5801-6e30-4fa0-9a95-6ebbd7622ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:20.559209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40741845-95cd-441d-8f96-ae0cb905885b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:20.616956 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117bf53-2a8e-4053-addd-e352ac149d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:20.618227 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117bf53-2a8e-4053-addd-e352ac149d4d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:20.618539 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117bf53-2a8e-4053-addd-e352ac149d4d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:20.618814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117bf53-2a8e-4053-addd-e352ac149d4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:20.631178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117bf53-2a8e-4053-addd-e352ac149d4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-9117bf53-2a8e-4053-addd-e352ac149d4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:20.636422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:20.645653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:20.649175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:20.649175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:20.649175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:20.649175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:20.649175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:20.649175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:20.649175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:20.649175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:20.649175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:20.649175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:20.649175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:20.649175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:20.649175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 157/302] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:20.649175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117bf53-2a8e-4053-addd-e352ac149d4d None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:20.651363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 146/303] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:20.655618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae5801-6e30-4fa0-9a95-6ebbd7622ca6 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:20.681991 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:20.718788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:20.792080 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae5801-6e30-4fa0-9a95-6ebbd7622ca6 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:20.795056 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117bf53-2a8e-4053-addd-e352ac149d4d None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:20.838138 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253db19-0b67-4123-bc6b-97948200b79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:20.838783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253db19-0b67-4123-bc6b-97948200b79d None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:20.839068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253db19-0b67-4123-bc6b-97948200b79d None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:20.839347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253db19-0b67-4123-bc6b-97948200b79d None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:20.840267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253db19-0b67-4123-bc6b-97948200b79d None admin] RBAC: Adding query filter params (name=member) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:20.840583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253db19-0b67-4123-bc6b-97948200b79d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:20.842174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253db19-0b67-4123-bc6b-97948200b79d None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:20.842804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e30378-fcad-40c2-935c-bd1fd6b084cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:20.843939 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e30378-fcad-40c2-935c-bd1fd6b084cf None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:20.843939 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e30378-fcad-40c2-935c-bd1fd6b084cf None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:20.844270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e30378-fcad-40c2-935c-bd1fd6b084cf None admin] PATH_INFO: `/v3/users/b3e6fa0069324aaca8d4ee660881eadb` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:20.861405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 147/304] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:20.861405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253db19-0b67-4123-bc6b-97948200b79d None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:20.869636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e30378-fcad-40c2-935c-bd1fd6b084cf None admin] RBAC: Authorizing `identity:get_user(user_id=b3e6fa0069324aaca8d4ee660881eadb)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:20.871346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e30378-fcad-40c2-935c-bd1fd6b084cf None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:20.873540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 158/305] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:20 2022] GET /identity/v3/users/b3e6fa0069324aaca8d4ee660881eadb => generated 293 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:20.878664 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e30378-fcad-40c2-935c-bd1fd6b084cf None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:20.981033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:21.015320 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253db19-0b67-4123-bc6b-97948200b79d None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:21.040814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e30378-fcad-40c2-935c-bd1fd6b084cf None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:21.045841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:21.062195 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09aed6-f9ca-494c-a7de-fd628cf6ea7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:21.062861 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09aed6-f9ca-494c-a7de-fd628cf6ea7a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:21.063144 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09aed6-f9ca-494c-a7de-fd628cf6ea7a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:21.063420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09aed6-f9ca-494c-a7de-fd628cf6ea7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:21.064779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09aed6-f9ca-494c-a7de-fd628cf6ea7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:21.066294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09aed6-f9ca-494c-a7de-fd628cf6ea7a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:21.068913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 148/306] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:40:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:21.069881 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09aed6-f9ca-494c-a7de-fd628cf6ea7a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:21.108739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35956e0c-4348-46af-8b5f-4d92873c3cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:21.109775 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35956e0c-4348-46af-8b5f-4d92873c3cc2 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:21.110165 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35956e0c-4348-46af-8b5f-4d92873c3cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:21.110546 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35956e0c-4348-46af-8b5f-4d92873c3cc2 None admin] PATH_INFO: `/v3/users/ae9e318c3bf7440f8b645235a8855ba6` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:21.112499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35956e0c-4348-46af-8b5f-4d92873c3cc2 None admin] RBAC: Authorizing `identity:get_user(user_id=ae9e318c3bf7440f8b645235a8855ba6)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:21.115972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35956e0c-4348-46af-8b5f-4d92873c3cc2 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:21.119030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 159/307] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:20 2022] GET /identity/v3/users/ae9e318c3bf7440f8b645235a8855ba6 => generated 293 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:21.120135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35956e0c-4348-46af-8b5f-4d92873c3cc2 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:21.205110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09aed6-f9ca-494c-a7de-fd628cf6ea7a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:21.258474 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e1fcd-0194-4cc2-8efd-02e4aebf2fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:21.258474 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e1fcd-0194-4cc2-8efd-02e4aebf2fb0 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:21.258474 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e1fcd-0194-4cc2-8efd-02e4aebf2fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:21.258474 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e1fcd-0194-4cc2-8efd-02e4aebf2fb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:21.259068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e1fcd-0194-4cc2-8efd-02e4aebf2fb0 None admin] RBAC: Adding query filter params (role.id=cf7c52610e954f74a30feba325390387, user.id=b3e6fa0069324aaca8d4ee660881eadb, scope.system=all) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:21.259068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e1fcd-0194-4cc2-8efd-02e4aebf2fb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:21.259342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e1fcd-0194-4cc2-8efd-02e4aebf2fb0 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:21.266955 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 149/308] 10.222.0.94 () {56 vars in 1308 bytes} [Tue Jul 26 14:40:21 2022] GET /identity/v3/role_assignments?user.id=b3e6fa0069324aaca8d4ee660881eadb&scope.system=all&role.id=cf7c52610e954f74a30feba325390387 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:21.266955 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e1fcd-0194-4cc2-8efd-02e4aebf2fb0 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:21.297701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35956e0c-4348-46af-8b5f-4d92873c3cc2 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:21.354301 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:21.381951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b7f93-701c-4f4c-8b87-addf6e97cdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:21.383493 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b7f93-701c-4f4c-8b87-addf6e97cdf2 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:21.384135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b7f93-701c-4f4c-8b87-addf6e97cdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:21.384595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b7f93-701c-4f4c-8b87-addf6e97cdf2 None admin] PATH_INFO: `/v3/groups` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:21.385879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b7f93-701c-4f4c-8b87-addf6e97cdf2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:21.388731 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b7f93-701c-4f4c-8b87-addf6e97cdf2 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:21.472785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 160/309] 10.222.0.94 () {60 vars in 1134 bytes} [Tue Jul 26 14:40:21 2022] POST /identity/v3/groups => generated 229 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:40:21.474687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0abbc-48ed-4a2e-86f9-f23e8d278b1a None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:21.475230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0abbc-48ed-4a2e-86f9-f23e8d278b1a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:21.475796 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0abbc-48ed-4a2e-86f9-f23e8d278b1a None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:21.478448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 161/310] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:21.488216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e4c2f-5d6e-411b-94a5-4eab199c7e24 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:21.489238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e4c2f-5d6e-411b-94a5-4eab199c7e24 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:21.490479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e4c2f-5d6e-411b-94a5-4eab199c7e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:21.504536 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e1fcd-0194-4cc2-8efd-02e4aebf2fb0 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:21.522381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-5c5e4c2f-5d6e-411b-94a5-4eab199c7e24 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:21.525510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5c5e4c2f-5d6e-411b-94a5-4eab199c7e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:21.589853 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:21.593195 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10854b-33d3-4500-98ed-d21a04e584a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:21.595287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10854b-33d3-4500-98ed-d21a04e584a6 None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:21.596327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10854b-33d3-4500-98ed-d21a04e584a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:21.597342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10854b-33d3-4500-98ed-d21a04e584a6 None admin] PATH_INFO: `/v3/system/users/ae9e318c3bf7440f8b645235a8855ba6/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:21.601840 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10854b-33d3-4500-98ed-d21a04e584a6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ae9e318c3bf7440f8b645235a8855ba6, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:21.605308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10854b-33d3-4500-98ed-d21a04e584a6 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:21.684540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 162/311] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:21.701739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 150/312] 10.222.0.94 () {58 vars in 1334 bytes} [Tue Jul 26 14:40:21 2022] PUT /identity/v3/system/users/ae9e318c3bf7440f8b645235a8855ba6/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:40:21.722193 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e4c2f-5d6e-411b-94a5-4eab199c7e24 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:21.882715 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e4c2f-5d6e-411b-94a5-4eab199c7e24 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:21.930392 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ecc81-4f75-4007-a796-ac80af49bc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:21.931118 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ecc81-4f75-4007-a796-ac80af49bc37 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:21.931428 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ecc81-4f75-4007-a796-ac80af49bc37 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:21.931706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ecc81-4f75-4007-a796-ac80af49bc37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:21.953835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ecc81-4f75-4007-a796-ac80af49bc37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-352ecc81-4f75-4007-a796-ac80af49bc37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:21.961858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:21.963636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:21.965132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:21.965132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:21.965132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:21.965132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:21.965132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:21.965132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:21.965132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:21.965132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:21.965132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:21.965132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:21.965132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:40:21.965132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:21.965132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 163/313] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:21.969558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10854b-33d3-4500-98ed-d21a04e584a6 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:22.123089 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:22.163247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10854b-33d3-4500-98ed-d21a04e584a6 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:22.214262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e2e5f-644a-4c3d-b84c-709e007f05f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:22.215016 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e2e5f-644a-4c3d-b84c-709e007f05f6 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:22.215354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e2e5f-644a-4c3d-b84c-709e007f05f6 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:22.215666 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e2e5f-644a-4c3d-b84c-709e007f05f6 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:22.216477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e2e5f-644a-4c3d-b84c-709e007f05f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:22.216785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e2e5f-644a-4c3d-b84c-709e007f05f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:22.218246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e2e5f-644a-4c3d-b84c-709e007f05f6 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:22.229883 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 151/314] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:22.235412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ecc81-4f75-4007-a796-ac80af49bc37 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:22.290299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:22.426818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ecc81-4f75-4007-a796-ac80af49bc37 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:22.494429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fa7c7-8940-4c1c-a727-40d2ae185d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:22.495576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fa7c7-8940-4c1c-a727-40d2ae185d00 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:22.496118 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fa7c7-8940-4c1c-a727-40d2ae185d00 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:22.496793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fa7c7-8940-4c1c-a727-40d2ae185d00 None admin] PATH_INFO: `/v3/users/248db7da4a88429fb74819edb303200f` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:22.499064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fa7c7-8940-4c1c-a727-40d2ae185d00 None admin] RBAC: Authorizing `identity:get_user(user_id=248db7da4a88429fb74819edb303200f)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:22.501051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fa7c7-8940-4c1c-a727-40d2ae185d00 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:22.503298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 164/315] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:22 2022] GET /identity/v3/users/248db7da4a88429fb74819edb303200f => generated 297 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:22.510051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e2e5f-644a-4c3d-b84c-709e007f05f6 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:22.643954 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e2e5f-644a-4c3d-b84c-709e007f05f6 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:22.686003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b63e9c-6953-4d9c-b364-f97b4d5c1734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:22.686543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b63e9c-6953-4d9c-b364-f97b4d5c1734 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:22.686826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b63e9c-6953-4d9c-b364-f97b4d5c1734 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:22.687223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b63e9c-6953-4d9c-b364-f97b4d5c1734 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:22.703886 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:22.775194 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b63e9c-6953-4d9c-b364-f97b4d5c1734 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:22.776974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b63e9c-6953-4d9c-b364-f97b4d5c1734 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:22.902888 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 152/316] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:22.909665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fa7c7-8940-4c1c-a727-40d2ae185d00 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:23.079153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:23.098150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fa7c7-8940-4c1c-a727-40d2ae185d00 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:23.157549 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384913e-acc8-495e-9207-0dbe8089bb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:23.158679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384913e-acc8-495e-9207-0dbe8089bb39 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:23.159144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384913e-acc8-495e-9207-0dbe8089bb39 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:23.159629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384913e-acc8-495e-9207-0dbe8089bb39 None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:23.162076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384913e-acc8-495e-9207-0dbe8089bb39 None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:23.164223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384913e-acc8-495e-9207-0dbe8089bb39 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:23.167034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 165/317] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:22 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:23.173948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b63e9c-6953-4d9c-b364-f97b4d5c1734 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:23.227873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:23.307668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b63e9c-6953-4d9c-b364-f97b4d5c1734 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:23.412263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce86cd-4421-4630-8f84-a2fd920f5214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:23.412877 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce86cd-4421-4630-8f84-a2fd920f5214 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:23.413164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce86cd-4421-4630-8f84-a2fd920f5214 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:23.413447 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce86cd-4421-4630-8f84-a2fd920f5214 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:23.414287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce86cd-4421-4630-8f84-a2fd920f5214 None admin] RBAC: Adding query filter params (role.id=cf7c52610e954f74a30feba325390387, scope.project.id=21ad3a8335ad492d858142d787614be0, user.id=248db7da4a88429fb74819edb303200f) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:23.414599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce86cd-4421-4630-8f84-a2fd920f5214 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:23.416147 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce86cd-4421-4630-8f84-a2fd920f5214 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:23.423613 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 153/318] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:40:23 2022] GET /identity/v3/role_assignments?user.id=248db7da4a88429fb74819edb303200f&scope.project.id=21ad3a8335ad492d858142d787614be0&role.id=cf7c52610e954f74a30feba325390387 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:23.987774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456fbd7-adf1-447d-a974-c64905fddc4b None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:23.988401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456fbd7-adf1-447d-a974-c64905fddc4b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:23.988401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456fbd7-adf1-447d-a974-c64905fddc4b None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:23.990832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 166/319] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:23.997002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e5a81-0c16-4b76-8f68-83af98dff311 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:23.997361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e5a81-0c16-4b76-8f68-83af98dff311 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:23.997580 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e5a81-0c16-4b76-8f68-83af98dff311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:24.025157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-2a5e5a81-0c16-4b76-8f68-83af98dff311 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:24.027055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-2a5e5a81-0c16-4b76-8f68-83af98dff311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:24.133855 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 154/320] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:24.157127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456fbd7-adf1-447d-a974-c64905fddc4b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:24.307733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456fbd7-adf1-447d-a974-c64905fddc4b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:24.357465 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98903275-1227-48d1-9ce2-cae6a8532016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:24.358222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98903275-1227-48d1-9ce2-cae6a8532016 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:24.358496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98903275-1227-48d1-9ce2-cae6a8532016 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:24.358795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98903275-1227-48d1-9ce2-cae6a8532016 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:24.360131 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98903275-1227-48d1-9ce2-cae6a8532016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:24.361614 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98903275-1227-48d1-9ce2-cae6a8532016 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:24.364690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 167/321] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:40:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:24.369099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bd634-e243-4855-93fe-93a27e040574 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:24.369881 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bd634-e243-4855-93fe-93a27e040574 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:24.370285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bd634-e243-4855-93fe-93a27e040574 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:24.372128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98903275-1227-48d1-9ce2-cae6a8532016 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:24.372883 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 155/322] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:24.382588 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418a431-1be8-43ef-b261-efe4616a9fc6 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:24.383079 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418a431-1be8-43ef-b261-efe4616a9fc6 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:24.383615 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418a431-1be8-43ef-b261-efe4616a9fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:24.412464 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:24.420266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-b418a431-1be8-43ef-b261-efe4616a9fc6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:24.422878 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b418a431-1be8-43ef-b261-efe4616a9fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:24.464934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:24.534846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98903275-1227-48d1-9ce2-cae6a8532016 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:24.556081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 156/323] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:40:24.587105 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418a431-1be8-43ef-b261-efe4616a9fc6 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:24.605197 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a69ce-a94b-4354-8881-564cdea31cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:24.606299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a69ce-a94b-4354-8881-564cdea31cfb None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:24.606804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a69ce-a94b-4354-8881-564cdea31cfb None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:24.607291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a69ce-a94b-4354-8881-564cdea31cfb None admin] PATH_INFO: `/v3/groups` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:24.608620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a69ce-a94b-4354-8881-564cdea31cfb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:24.611383 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a69ce-a94b-4354-8881-564cdea31cfb None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:24.744110 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 168/324] 10.222.0.94 () {60 vars in 1134 bytes} [Tue Jul 26 14:40:24 2022] POST /identity/v3/groups => generated 226 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:40:24.770558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418a431-1be8-43ef-b261-efe4616a9fc6 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:24.815364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb4987-6a3f-4a08-8ab7-9f37f60a2518 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:24.815839 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb4987-6a3f-4a08-8ab7-9f37f60a2518 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:24.815839 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb4987-6a3f-4a08-8ab7-9f37f60a2518 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:24.818126 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 169/325] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:40:24.825236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f17369-117e-4ca2-969c-77921e287ced None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:24.825605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f17369-117e-4ca2-969c-77921e287ced None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:24.825969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f17369-117e-4ca2-969c-77921e287ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:24.849862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe54a8-6676-45ae-b629-d7c565097128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:24.850539 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe54a8-6676-45ae-b629-d7c565097128 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:24.850818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe54a8-6676-45ae-b629-d7c565097128 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:24.851095 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe54a8-6676-45ae-b629-d7c565097128 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:24.857542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-20f17369-117e-4ca2-969c-77921e287ced None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:24.859721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-20f17369-117e-4ca2-969c-77921e287ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:24.865747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe54a8-6676-45ae-b629-d7c565097128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-4ffe54a8-6676-45ae-b629-d7c565097128 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:24.871121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:24.872624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:24.873941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:24.873941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:24.873941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:24.873941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:24.873941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:24.873941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:24.873941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:24.873941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:24.873941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:24.873941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:24.873941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:40:24.873941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:24.873941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 157/326] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:40:24.879957 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe54a8-6676-45ae-b629-d7c565097128 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:24.937481 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:24.991533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 170/327] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:40:25.029729 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f17369-117e-4ca2-969c-77921e287ced None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:25.067241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:25.077697 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe54a8-6676-45ae-b629-d7c565097128 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:25.144443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db11e1-4cc1-40d6-b0cb-481328622f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:25.146162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db11e1-4cc1-40d6-b0cb-481328622f11 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:25.146802 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db11e1-4cc1-40d6-b0cb-481328622f11 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:25.147711 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db11e1-4cc1-40d6-b0cb-481328622f11 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:25.148454 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db11e1-4cc1-40d6-b0cb-481328622f11 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:25.148946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db11e1-4cc1-40d6-b0cb-481328622f11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:25.151627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db11e1-4cc1-40d6-b0cb-481328622f11 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:25.175025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 158/328] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:25.188608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db11e1-4cc1-40d6-b0cb-481328622f11 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:25.208768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f17369-117e-4ca2-969c-77921e287ced None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:25.254899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8424b-1f9b-46bc-8c85-18a2e6c1a844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:25.254899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8424b-1f9b-46bc-8c85-18a2e6c1a844 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:25.254899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8424b-1f9b-46bc-8c85-18a2e6c1a844 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:25.254899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8424b-1f9b-46bc-8c85-18a2e6c1a844 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:25.280935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8424b-1f9b-46bc-8c85-18a2e6c1a844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-0ea8424b-1f9b-46bc-8c85-18a2e6c1a844 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:25.287170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:25.289144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:25.290797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:25.290797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:25.290797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:25.290797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:25.290797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:25.290797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:25.290797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:25.290797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:25.290797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:25.290797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:25.290797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:25.290797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:25.290797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 171/329] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:40:25.299812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8424b-1f9b-46bc-8c85-18a2e6c1a844 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:25.373161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:25.406173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db11e1-4cc1-40d6-b0cb-481328622f11 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:25.411032 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:25.445348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8424b-1f9b-46bc-8c85-18a2e6c1a844 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:25.478080 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c7dfc-b958-445f-8c02-133e1accfe9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:25.479432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c7dfc-b958-445f-8c02-133e1accfe9f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:25.480130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c7dfc-b958-445f-8c02-133e1accfe9f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:25.482153 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c7dfc-b958-445f-8c02-133e1accfe9f None admin] PATH_INFO: `/v3/users/b3e6fa0069324aaca8d4ee660881eadb` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:25.484525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c7dfc-b958-445f-8c02-133e1accfe9f None admin] RBAC: Authorizing `identity:get_user(user_id=b3e6fa0069324aaca8d4ee660881eadb)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:25.487139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c7dfc-b958-445f-8c02-133e1accfe9f None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:25.491717 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 159/330] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:25 2022] GET /identity/v3/users/b3e6fa0069324aaca8d4ee660881eadb => generated 293 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:25.499646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505d2f0-a837-4313-8b84-228706f6e2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:25.500403 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505d2f0-a837-4313-8b84-228706f6e2cf None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:25.500677 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505d2f0-a837-4313-8b84-228706f6e2cf None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:25.500952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505d2f0-a837-4313-8b84-228706f6e2cf None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:25.501693 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505d2f0-a837-4313-8b84-228706f6e2cf None admin] RBAC: Adding query filter params (name=member) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:25.502000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505d2f0-a837-4313-8b84-228706f6e2cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:25.502139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c7dfc-b958-445f-8c02-133e1accfe9f None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:25.503573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505d2f0-a837-4313-8b84-228706f6e2cf None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:25.516475 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 172/331] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:25.528740 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505d2f0-a837-4313-8b84-228706f6e2cf None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:25.673937 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505d2f0-a837-4313-8b84-228706f6e2cf None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:25.717931 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c7dfc-b958-445f-8c02-133e1accfe9f None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:25.725435 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e838d80-c5fb-495d-b244-b31f3d79bd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:25.725847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e838d80-c5fb-495d-b244-b31f3d79bd0a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:25.726397 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e838d80-c5fb-495d-b244-b31f3d79bd0a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:25.727028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e838d80-c5fb-495d-b244-b31f3d79bd0a None admin] PATH_INFO: `/v3/users/ae9e318c3bf7440f8b645235a8855ba6` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:25.729128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e838d80-c5fb-495d-b244-b31f3d79bd0a None admin] RBAC: Authorizing `identity:get_user(user_id=ae9e318c3bf7440f8b645235a8855ba6)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:25.731182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e838d80-c5fb-495d-b244-b31f3d79bd0a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:25.733577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 173/332] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:25 2022] GET /identity/v3/users/ae9e318c3bf7440f8b645235a8855ba6 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:25.742045 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e838d80-c5fb-495d-b244-b31f3d79bd0a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:25.767033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:25.795153 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9abea7-2251-46d3-8012-4cf9651ea1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:25.796482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9abea7-2251-46d3-8012-4cf9651ea1bc None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:25.797094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9abea7-2251-46d3-8012-4cf9651ea1bc None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:25.797865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9abea7-2251-46d3-8012-4cf9651ea1bc None admin] PATH_INFO: `/v3/system/users/b3e6fa0069324aaca8d4ee660881eadb/roles/cf7c52610e954f74a30feba325390387` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:25.801260 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9abea7-2251-46d3-8012-4cf9651ea1bc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b3e6fa0069324aaca8d4ee660881eadb, role_id=cf7c52610e954f74a30feba325390387)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:25.804601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9abea7-2251-46d3-8012-4cf9651ea1bc None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:25.901036 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 160/333] 10.222.0.94 () {58 vars in 1334 bytes} [Tue Jul 26 14:40:25 2022] PUT /identity/v3/system/users/b3e6fa0069324aaca8d4ee660881eadb/roles/cf7c52610e954f74a30feba325390387 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:40:25.998302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e838d80-c5fb-495d-b244-b31f3d79bd0a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:26.051235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7be3e-60b2-4432-966d-f4996c234d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:26.052150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7be3e-60b2-4432-966d-f4996c234d44 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:26.052590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7be3e-60b2-4432-966d-f4996c234d44 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:26.052874 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7be3e-60b2-4432-966d-f4996c234d44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:26.053854 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7be3e-60b2-4432-966d-f4996c234d44 None admin] RBAC: Adding query filter params (role.id=6a924f1d619643b3be92bab6fddb3eb2, user.id=ae9e318c3bf7440f8b645235a8855ba6, scope.system=all) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:26.054225 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7be3e-60b2-4432-966d-f4996c234d44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:26.055915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7be3e-60b2-4432-966d-f4996c234d44 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:26.072826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 174/334] 10.222.0.94 () {56 vars in 1308 bytes} [Tue Jul 26 14:40:25 2022] GET /identity/v3/role_assignments?user.id=ae9e318c3bf7440f8b645235a8855ba6&scope.system=all&role.id=6a924f1d619643b3be92bab6fddb3eb2 => generated 510 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:27.995090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6966fa-5044-4ff6-bee0-0e3dff362382 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:27.996611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6966fa-5044-4ff6-bee0-0e3dff362382 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:27.997304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6966fa-5044-4ff6-bee0-0e3dff362382 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:28.000631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 161/335] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:28.009824 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93832e4a-6553-44b6-92e3-4cd81ca4d078 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:28.010695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93832e4a-6553-44b6-92e3-4cd81ca4d078 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:28.010695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93832e4a-6553-44b6-92e3-4cd81ca4d078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:28.057427 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-93832e4a-6553-44b6-92e3-4cd81ca4d078 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:28.059784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-93832e4a-6553-44b6-92e3-4cd81ca4d078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:28.199294 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:28.216045 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 175/336] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:28.271496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6966fa-5044-4ff6-bee0-0e3dff362382 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:28.308945 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:28.525116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6966fa-5044-4ff6-bee0-0e3dff362382 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:28.607910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526a145-38c4-4d4e-a8f3-ae714e6c0843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:28.609045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526a145-38c4-4d4e-a8f3-ae714e6c0843 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:28.609539 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526a145-38c4-4d4e-a8f3-ae714e6c0843 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:28.610029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526a145-38c4-4d4e-a8f3-ae714e6c0843 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:28.641383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526a145-38c4-4d4e-a8f3-ae714e6c0843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-c526a145-38c4-4d4e-a8f3-ae714e6c0843 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:28.653249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:28.654326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:28.655191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:28.655191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:28.655191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:28.655191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:28.655191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:28.655191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:28.655191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:28.655191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:28.655191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:28.655191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:28.655191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:28.655191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:28.655812 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 162/337] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:28.666887 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93832e4a-6553-44b6-92e3-4cd81ca4d078 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:28.885982 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93832e4a-6553-44b6-92e3-4cd81ca4d078 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:28.955825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023d6aa-25ea-42cf-8544-c5a2d75a174f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:28.957780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023d6aa-25ea-42cf-8544-c5a2d75a174f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:28.958287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023d6aa-25ea-42cf-8544-c5a2d75a174f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:28.958776 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023d6aa-25ea-42cf-8544-c5a2d75a174f None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:28.960200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023d6aa-25ea-42cf-8544-c5a2d75a174f None admin] RBAC: Adding query filter params (name=member) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:28.960725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023d6aa-25ea-42cf-8544-c5a2d75a174f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:28.963260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023d6aa-25ea-42cf-8544-c5a2d75a174f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:28.980511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 176/338] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:28.990291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526a145-38c4-4d4e-a8f3-ae714e6c0843 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:29.176701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:29.237728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526a145-38c4-4d4e-a8f3-ae714e6c0843 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:29.338493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b30a3-938e-44ab-b658-04943fd4f9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:29.339616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b30a3-938e-44ab-b658-04943fd4f9a4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:29.340122 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b30a3-938e-44ab-b658-04943fd4f9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:29.340558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b30a3-938e-44ab-b658-04943fd4f9a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:29.366863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7910cb-a5f4-455e-8f46-bf2d2952ef27 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:29.367425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7910cb-a5f4-455e-8f46-bf2d2952ef27 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:29.367922 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7910cb-a5f4-455e-8f46-bf2d2952ef27 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:29.371054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 177/339] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:40:29.385333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98352d3a-3ff5-4cb5-b85d-2e887b520970 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:29.386382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98352d3a-3ff5-4cb5-b85d-2e887b520970 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:29.387164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98352d3a-3ff5-4cb5-b85d-2e887b520970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:29.437721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-98352d3a-3ff5-4cb5-b85d-2e887b520970 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:29.440707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-98352d3a-3ff5-4cb5-b85d-2e887b520970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:29.507538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b30a3-938e-44ab-b658-04943fd4f9a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:29.509436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b30a3-938e-44ab-b658-04943fd4f9a4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:29.593662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:29.643530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 178/340] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:29.655797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533afbf-93f7-4473-84c5-62f6d92941d8 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:29.656288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533afbf-93f7-4473-84c5-62f6d92941d8 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:29.656750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533afbf-93f7-4473-84c5-62f6d92941d8 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:29.659579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 179/341] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:29.671963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb370f6-1605-4232-8fd7-e5940bacd3f5 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:29.672436 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb370f6-1605-4232-8fd7-e5940bacd3f5 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:29.672974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb370f6-1605-4232-8fd7-e5940bacd3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:29.702822 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-2eb370f6-1605-4232-8fd7-e5940bacd3f5 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:29.704008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2eb370f6-1605-4232-8fd7-e5940bacd3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:29.739665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:29.808274 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 163/342] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 819 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:29.812042 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b30a3-938e-44ab-b658-04943fd4f9a4 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:29.894493 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 180/343] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:29.897501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb370f6-1605-4232-8fd7-e5940bacd3f5 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:30.014234 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b30a3-938e-44ab-b658-04943fd4f9a4 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:30.090864 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b835e-84de-40ac-8b22-4bddd753c7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:30.093006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b835e-84de-40ac-8b22-4bddd753c7b7 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:30.093750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b835e-84de-40ac-8b22-4bddd753c7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:30.094958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b835e-84de-40ac-8b22-4bddd753c7b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:30.115102 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b835e-84de-40ac-8b22-4bddd753c7b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-229b835e-84de-40ac-8b22-4bddd753c7b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:30.121644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:30.123223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:30.124999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:30.124999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:30.124999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:30.124999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:30.124999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:30.124999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:30.124999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:30.124999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:30.124999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:30.124999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:30.124999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:30.124999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:30.124999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 164/344] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:30.127133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b835e-84de-40ac-8b22-4bddd753c7b7 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:30.132394 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb370f6-1605-4232-8fd7-e5940bacd3f5 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:30.160119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:30.205189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:30.208115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58bde4-726a-4bc3-9922-664f30ac1f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:30.209295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58bde4-726a-4bc3-9922-664f30ac1f0f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:30.209605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58bde4-726a-4bc3-9922-664f30ac1f0f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:30.209862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58bde4-726a-4bc3-9922-664f30ac1f0f None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:30.211184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58bde4-726a-4bc3-9922-664f30ac1f0f None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:30.213005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58bde4-726a-4bc3-9922-664f30ac1f0f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:30.217020 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 181/345] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:29 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:30.218485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015a2ef-5d5c-4c96-ab46-34992b795535 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:30.218798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015a2ef-5d5c-4c96-ab46-34992b795535 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:30.219035 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015a2ef-5d5c-4c96-ab46-34992b795535 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:30.220473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 182/346] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:30.221678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc771385-21cc-4e82-be52-be9fdb234163 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:30.221954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc771385-21cc-4e82-be52-be9fdb234163 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:30.222223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc771385-21cc-4e82-be52-be9fdb234163 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:30.224126 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 183/347] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:30.225509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc771385-21cc-4e82-be52-be9fdb234163 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:30.367602 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b835e-84de-40ac-8b22-4bddd753c7b7 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:30.416432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc771385-21cc-4e82-be52-be9fdb234163 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:30.438433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2fa30-da8c-4ad5-a763-611fe7e788db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:30.440197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2fa30-da8c-4ad5-a763-611fe7e788db None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:30.441112 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2fa30-da8c-4ad5-a763-611fe7e788db None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:30.442185 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2fa30-da8c-4ad5-a763-611fe7e788db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:30.464520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2fa30-da8c-4ad5-a763-611fe7e788db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-ecc2fa30-da8c-4ad5-a763-611fe7e788db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:30.474691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:30.476380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:30.477761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:30.477761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:30.477761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:30.477761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:30.477761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:30.477761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:30.477761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:30.477761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:30.477761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:30.477761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:30.477761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:30.477761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:30.477761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 165/348] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:40:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:30.481320 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2fa30-da8c-4ad5-a763-611fe7e788db None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:30.488533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d244c0e-ca6b-4e19-b618-95791cd8e221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:30.489608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d244c0e-ca6b-4e19-b618-95791cd8e221 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:30.490001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d244c0e-ca6b-4e19-b618-95791cd8e221 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:30.490291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d244c0e-ca6b-4e19-b618-95791cd8e221 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:30.491222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d244c0e-ca6b-4e19-b618-95791cd8e221 None admin] RBAC: Adding query filter params (name=member) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:30.492346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d244c0e-ca6b-4e19-b618-95791cd8e221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:30.494287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d244c0e-ca6b-4e19-b618-95791cd8e221 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:30.508448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 184/349] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:30.510209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d1657-6ae1-4fdc-8153-0cb7087c4d77 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:30.510632 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d1657-6ae1-4fdc-8153-0cb7087c4d77 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:30.511005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d1657-6ae1-4fdc-8153-0cb7087c4d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:30.534722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-002d1657-6ae1-4fdc-8153-0cb7087c4d77 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:30.536462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-002d1657-6ae1-4fdc-8153-0cb7087c4d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:30.627408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:30.665136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 185/350] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:30.667505 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29920f43-6434-4577-ab82-dc8856846f04 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:30.667991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29920f43-6434-4577-ab82-dc8856846f04 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:30.668300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29920f43-6434-4577-ab82-dc8856846f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:30.693151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-29920f43-6434-4577-ab82-dc8856846f04 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:30.694789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-29920f43-6434-4577-ab82-dc8856846f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:30.695787 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:30.716968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2fa30-da8c-4ad5-a763-611fe7e788db None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:30.801328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4f320-e2bf-4c29-947b-af0ff6c751dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:30.802482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4f320-e2bf-4c29-947b-af0ff6c751dc None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:30.802967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4f320-e2bf-4c29-947b-af0ff6c751dc None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:30.803679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4f320-e2bf-4c29-947b-af0ff6c751dc None admin] PATH_INFO: `/v3/groups/2960f6ca0176451eb54f65211c582146` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:30.815197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4f320-e2bf-4c29-947b-af0ff6c751dc None admin] RBAC: Authorizing `identity:get_group(group_id=2960f6ca0176451eb54f65211c582146)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:30.818494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4f320-e2bf-4c29-947b-af0ff6c751dc None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:30.822161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 166/351] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:30 2022] GET /identity/v3/groups/2960f6ca0176451eb54f65211c582146 => generated 226 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:30.824027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4f320-e2bf-4c29-947b-af0ff6c751dc None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:30.856350 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 186/352] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:30.857911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29920f43-6434-4577-ab82-dc8856846f04 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:31.034165 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4f320-e2bf-4c29-947b-af0ff6c751dc None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:31.049535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29920f43-6434-4577-ab82-dc8856846f04 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:31.059902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:31.107211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1e29a-bb18-4e8c-b021-9e30e196c5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:31.107786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2af13-d0a1-40bb-b82b-1adc84226133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:31.108270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1e29a-bb18-4e8c-b021-9e30e196c5c4 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:31.108441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2af13-d0a1-40bb-b82b-1adc84226133 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:31.108703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2af13-d0a1-40bb-b82b-1adc84226133 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:31.108940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1e29a-bb18-4e8c-b021-9e30e196c5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:31.109050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2af13-d0a1-40bb-b82b-1adc84226133 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:31.109726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2af13-d0a1-40bb-b82b-1adc84226133 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:31.109953 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1e29a-bb18-4e8c-b021-9e30e196c5c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:31.110064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2af13-d0a1-40bb-b82b-1adc84226133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:31.111510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2af13-d0a1-40bb-b82b-1adc84226133 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:31.124107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 167/353] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:40:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:31.125252 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2af13-d0a1-40bb-b82b-1adc84226133 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:31.172374 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:31.264317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1e29a-bb18-4e8c-b021-9e30e196c5c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:31.266735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1e29a-bb18-4e8c-b021-9e30e196c5c4 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:31.375544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2af13-d0a1-40bb-b82b-1adc84226133 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:31.455928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e01395-6037-40d4-bff3-92360fa73423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:31.457042 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e01395-6037-40d4-bff3-92360fa73423 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:31.457529 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e01395-6037-40d4-bff3-92360fa73423 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:31.458156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e01395-6037-40d4-bff3-92360fa73423 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:31.479357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e01395-6037-40d4-bff3-92360fa73423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-a1e01395-6037-40d4-bff3-92360fa73423 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:31.491728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:31.493354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:31.494752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:31.494752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:31.494752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:31.494752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:31.494752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:31.494752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:31.494752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:31.494752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:31.494752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:31.494752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:31.494752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:31.494752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:31.494752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 168/354] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:40:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:31.498957 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e01395-6037-40d4-bff3-92360fa73423 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:31.499211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 187/355] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:31.500496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e03b33-6a1f-4dbc-b1ca-eefeb0f785fa None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:31.500914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e03b33-6a1f-4dbc-b1ca-eefeb0f785fa None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:31.501331 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e03b33-6a1f-4dbc-b1ca-eefeb0f785fa None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:31.503353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 188/356] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:31.505845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e03b33-6a1f-4dbc-b1ca-eefeb0f785fa None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:31.522966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:31.692678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:31.733196 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e01395-6037-40d4-bff3-92360fa73423 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:31.736143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e03b33-6a1f-4dbc-b1ca-eefeb0f785fa None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:31.805189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49318b98-57c5-4d81-9bea-844a3999bdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:31.806308 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49318b98-57c5-4d81-9bea-844a3999bdf2 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:31.806803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49318b98-57c5-4d81-9bea-844a3999bdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:31.807258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49318b98-57c5-4d81-9bea-844a3999bdf2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:31.815559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fbe65-3451-412b-92b8-9a40eef37bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:31.817387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fbe65-3451-412b-92b8-9a40eef37bc3 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:31.818097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fbe65-3451-412b-92b8-9a40eef37bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:31.818591 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fbe65-3451-412b-92b8-9a40eef37bc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:31.823674 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fbe65-3451-412b-92b8-9a40eef37bc3 None admin] RBAC: Adding query filter params (group.id=2960f6ca0176451eb54f65211c582146, role.id=6a924f1d619643b3be92bab6fddb3eb2, scope.project.id=d0c721e2529541e5966d3c21426e447a) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:31.823674 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fbe65-3451-412b-92b8-9a40eef37bc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:31.825840 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fbe65-3451-412b-92b8-9a40eef37bc3 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:31.830107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49318b98-57c5-4d81-9bea-844a3999bdf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:31.837810 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 169/357] 10.222.0.94 () {56 vars in 1376 bytes} [Tue Jul 26 14:40:31 2022] GET /identity/v3/role_assignments?group.id=2960f6ca0176451eb54f65211c582146&scope.project.id=d0c721e2529541e5966d3c21426e447a&role.id=6a924f1d619643b3be92bab6fddb3eb2 => generated 260 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:31.838869 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fbe65-3451-412b-92b8-9a40eef37bc3 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-49318b98-57c5-4d81-9bea-844a3999bdf2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:31.850569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:31.852485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:31.854234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:31.854234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:31.854234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:31.854234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:31.854234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:31.854234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:31.854234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:31.854234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:31.854234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:31.854234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:31.854234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 14:40:31.854234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:31.854234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 189/358] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:31.857940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49318b98-57c5-4d81-9bea-844a3999bdf2 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:31.998212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:32.080709 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49318b98-57c5-4d81-9bea-844a3999bdf2 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:32.081126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fbe65-3451-412b-92b8-9a40eef37bc3 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:32.158671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1111d5-7bd8-46e8-b083-79e73d6e2169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:32.160139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1111d5-7bd8-46e8-b083-79e73d6e2169 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:32.160139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1111d5-7bd8-46e8-b083-79e73d6e2169 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:32.160602 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d18624-73f8-46e3-a7b6-b8934f4646b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:32.160780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1111d5-7bd8-46e8-b083-79e73d6e2169 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:32.161212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d18624-73f8-46e3-a7b6-b8934f4646b9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:32.161787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d18624-73f8-46e3-a7b6-b8934f4646b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:32.162102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d18624-73f8-46e3-a7b6-b8934f4646b9 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:32.162870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d18624-73f8-46e3-a7b6-b8934f4646b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:32.163180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d18624-73f8-46e3-a7b6-b8934f4646b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:32.164906 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d18624-73f8-46e3-a7b6-b8934f4646b9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:32.178176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 190/359] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:40:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:32.178461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d18624-73f8-46e3-a7b6-b8934f4646b9 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:32.200383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:32.302137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1111d5-7bd8-46e8-b083-79e73d6e2169 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:32.305002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1111d5-7bd8-46e8-b083-79e73d6e2169 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:32.364969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d18624-73f8-46e3-a7b6-b8934f4646b9 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:32.400540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:32.412913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc959d0e-977c-458f-ad85-eb60c5ef0b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:32.413989 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc959d0e-977c-458f-ad85-eb60c5ef0b3b None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:32.414479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc959d0e-977c-458f-ad85-eb60c5ef0b3b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:32.414959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc959d0e-977c-458f-ad85-eb60c5ef0b3b None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:32.417307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc959d0e-977c-458f-ad85-eb60c5ef0b3b None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:32.419347 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc959d0e-977c-458f-ad85-eb60c5ef0b3b None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:32.427138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 191/360] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:32 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:32.427138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e743d19-94ca-483f-ac40-5b64a3104822 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:32.427138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e743d19-94ca-483f-ac40-5b64a3104822 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:32.427138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e743d19-94ca-483f-ac40-5b64a3104822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:32.458188 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-4e743d19-94ca-483f-ac40-5b64a3104822 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:32.460360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4e743d19-94ca-483f-ac40-5b64a3104822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:32.482044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 170/361] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:32.484497 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1111d5-7bd8-46e8-b083-79e73d6e2169 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:32.558142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:32.582721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 192/362] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:32.583504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e743d19-94ca-483f-ac40-5b64a3104822 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:32.668919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1111d5-7bd8-46e8-b083-79e73d6e2169 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:32.719899 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cd108-4dc3-4ce6-993a-ba6939501505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:32.721192 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cd108-4dc3-4ce6-993a-ba6939501505 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:32.721624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cd108-4dc3-4ce6-993a-ba6939501505 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:32.722011 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cd108-4dc3-4ce6-993a-ba6939501505 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:32.723900 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cd108-4dc3-4ce6-993a-ba6939501505 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:32.724351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cd108-4dc3-4ce6-993a-ba6939501505 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:32.726478 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cd108-4dc3-4ce6-993a-ba6939501505 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:32.741806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 171/363] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:32.742734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cd108-4dc3-4ce6-993a-ba6939501505 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:32.760833 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e743d19-94ca-483f-ac40-5b64a3104822 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:32.808622 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:32.826361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1eb83-29b2-4e77-a51c-64c56dcb439a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:32.826824 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1eb83-29b2-4e77-a51c-64c56dcb439a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:32.827259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1eb83-29b2-4e77-a51c-64c56dcb439a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:32.827880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1eb83-29b2-4e77-a51c-64c56dcb439a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:32.955856 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cd108-4dc3-4ce6-993a-ba6939501505 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:32.971335 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:32.981304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1eb83-29b2-4e77-a51c-64c56dcb439a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:32.984251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1eb83-29b2-4e77-a51c-64c56dcb439a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:33.008531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3cc9b-0021-4365-9738-024e616afd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:33.009202 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3cc9b-0021-4365-9738-024e616afd21 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:33.009486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3cc9b-0021-4365-9738-024e616afd21 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:33.009771 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3cc9b-0021-4365-9738-024e616afd21 None admin] PATH_INFO: `/v3/groups/2960f6ca0176451eb54f65211c582146` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:33.011737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3cc9b-0021-4365-9738-024e616afd21 None admin] RBAC: Authorizing `identity:get_group(group_id=2960f6ca0176451eb54f65211c582146)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:33.014549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3cc9b-0021-4365-9738-024e616afd21 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:33.027203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 172/364] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:32 2022] GET /identity/v3/groups/2960f6ca0176451eb54f65211c582146 => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:33.028436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3cc9b-0021-4365-9738-024e616afd21 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:33.195961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 193/365] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:33.198026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1eb83-29b2-4e77-a51c-64c56dcb439a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:33.239876 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3cc9b-0021-4365-9738-024e616afd21 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:33.260424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:33.329654 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86464c4-ef4f-41dc-a843-85d7fe686f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:33.330720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86464c4-ef4f-41dc-a843-85d7fe686f4d None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:33.331152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86464c4-ef4f-41dc-a843-85d7fe686f4d None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:33.331646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86464c4-ef4f-41dc-a843-85d7fe686f4d None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:33.334392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86464c4-ef4f-41dc-a843-85d7fe686f4d None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:33.337231 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86464c4-ef4f-41dc-a843-85d7fe686f4d None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:33.341395 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 173/366] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:33 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:33.348375 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86464c4-ef4f-41dc-a843-85d7fe686f4d None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:33.434623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1eb83-29b2-4e77-a51c-64c56dcb439a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:33.445789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:33.502020 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c422ad-e584-4411-aec6-bd24cc4288b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:33.502787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c422ad-e584-4411-aec6-bd24cc4288b5 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:33.503242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c422ad-e584-4411-aec6-bd24cc4288b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:33.504299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c422ad-e584-4411-aec6-bd24cc4288b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:33.523186 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c422ad-e584-4411-aec6-bd24cc4288b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-92c422ad-e584-4411-aec6-bd24cc4288b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:33.529629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:33.531504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:33.533137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:33.533137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:33.533137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:33.533137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:33.533137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:33.533137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:33.533137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:33.533137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:33.533137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:33.533137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:33.533137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:40:33.533137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:33.540728 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 194/367] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:40:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:33.542453 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c422ad-e584-4411-aec6-bd24cc4288b5 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:33.563304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86464c4-ef4f-41dc-a843-85d7fe686f4d None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:33.638558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0963775-e164-4494-81e2-6f840ab2d53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:33.639726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0963775-e164-4494-81e2-6f840ab2d53a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:33.640214 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0963775-e164-4494-81e2-6f840ab2d53a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:33.640769 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0963775-e164-4494-81e2-6f840ab2d53a None admin] PATH_INFO: `/v3/users/b3e6fa0069324aaca8d4ee660881eadb` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:33.643066 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0963775-e164-4494-81e2-6f840ab2d53a None admin] RBAC: Authorizing `identity:get_user(user_id=b3e6fa0069324aaca8d4ee660881eadb)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:33.647087 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0963775-e164-4494-81e2-6f840ab2d53a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:33.650657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 174/368] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:40:33 2022] GET /identity/v3/users/b3e6fa0069324aaca8d4ee660881eadb => generated 293 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:33.660569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0963775-e164-4494-81e2-6f840ab2d53a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:33.684530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:33.719001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c422ad-e584-4411-aec6-bd24cc4288b5 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:33.799551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa491bf1-a4fc-4894-b16d-10e5b65a4cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:33.800634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa491bf1-a4fc-4894-b16d-10e5b65a4cac None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:33.801146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa491bf1-a4fc-4894-b16d-10e5b65a4cac None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:33.801477 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa491bf1-a4fc-4894-b16d-10e5b65a4cac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:33.804048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa491bf1-a4fc-4894-b16d-10e5b65a4cac None admin] RBAC: Adding query filter params (group.id=2960f6ca0176451eb54f65211c582146, role.id=6a924f1d619643b3be92bab6fddb3eb2, scope.project.id=21ad3a8335ad492d858142d787614be0) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:33.804589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa491bf1-a4fc-4894-b16d-10e5b65a4cac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:33.807742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa491bf1-a4fc-4894-b16d-10e5b65a4cac None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:33.821229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 195/369] 10.222.0.94 () {56 vars in 1376 bytes} [Tue Jul 26 14:40:33 2022] GET /identity/v3/role_assignments?group.id=2960f6ca0176451eb54f65211c582146&scope.project.id=21ad3a8335ad492d858142d787614be0&role.id=6a924f1d619643b3be92bab6fddb3eb2 => generated 260 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:33.822895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa491bf1-a4fc-4894-b16d-10e5b65a4cac None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:33.878909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0963775-e164-4494-81e2-6f840ab2d53a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:33.908961 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:33.956260 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599da95-4faf-45f3-95ae-e43ad541fdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:33.958911 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599da95-4faf-45f3-95ae-e43ad541fdc3 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:33.959965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599da95-4faf-45f3-95ae-e43ad541fdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:33.962314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599da95-4faf-45f3-95ae-e43ad541fdc3 None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:33.965339 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599da95-4faf-45f3-95ae-e43ad541fdc3 None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:33.968773 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599da95-4faf-45f3-95ae-e43ad541fdc3 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:33.973452 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 175/370] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:33 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:33.975326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599da95-4faf-45f3-95ae-e43ad541fdc3 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:33.978157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa491bf1-a4fc-4894-b16d-10e5b65a4cac None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:34.045431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67afe0-fec2-47a6-b2ca-800e6adc8056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:34.046550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67afe0-fec2-47a6-b2ca-800e6adc8056 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:34.047028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67afe0-fec2-47a6-b2ca-800e6adc8056 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:34.047534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67afe0-fec2-47a6-b2ca-800e6adc8056 None admin] PATH_INFO: `/v3/groups/2960f6ca0176451eb54f65211c582146` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:34.049895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67afe0-fec2-47a6-b2ca-800e6adc8056 None admin] RBAC: Authorizing `identity:get_group(group_id=2960f6ca0176451eb54f65211c582146)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:34.053214 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67afe0-fec2-47a6-b2ca-800e6adc8056 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:34.057028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 196/371] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:33 2022] GET /identity/v3/groups/2960f6ca0176451eb54f65211c582146 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:34.057733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67afe0-fec2-47a6-b2ca-800e6adc8056 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:34.095849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:34.205729 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599da95-4faf-45f3-95ae-e43ad541fdc3 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:34.288976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e1240-93f3-4929-a591-b9b4c1ced93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:34.290006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e1240-93f3-4929-a591-b9b4c1ced93e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:34.290486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e1240-93f3-4929-a591-b9b4c1ced93e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:34.290958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e1240-93f3-4929-a591-b9b4c1ced93e None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:34.292280 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e1240-93f3-4929-a591-b9b4c1ced93e None admin] RBAC: Adding query filter params (name=admin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:34.293146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e1240-93f3-4929-a591-b9b4c1ced93e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:34.296467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e1240-93f3-4929-a591-b9b4c1ced93e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:34.301441 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67afe0-fec2-47a6-b2ca-800e6adc8056 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:34.323707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 176/372] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:34.327688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e1240-93f3-4929-a591-b9b4c1ced93e None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:34.374925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1d929-4170-487d-a04d-91e7a101baf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:34.375977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1d929-4170-487d-a04d-91e7a101baf8 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:34.376442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1d929-4170-487d-a04d-91e7a101baf8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:34.376904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1d929-4170-487d-a04d-91e7a101baf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:34.378871 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1d929-4170-487d-a04d-91e7a101baf8 None admin] RBAC: Adding query filter params (role.id=cf7c52610e954f74a30feba325390387, user.id=b3e6fa0069324aaca8d4ee660881eadb, scope.system=all) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:34.379032 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1d929-4170-487d-a04d-91e7a101baf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:34.381481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1d929-4170-487d-a04d-91e7a101baf8 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:34.405238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 197/373] 10.222.0.94 () {56 vars in 1308 bytes} [Tue Jul 26 14:40:34 2022] GET /identity/v3/role_assignments?user.id=b3e6fa0069324aaca8d4ee660881eadb&scope.system=all&role.id=cf7c52610e954f74a30feba325390387 => generated 510 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:34.406426 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1d929-4170-487d-a04d-91e7a101baf8 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:34.414111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:34.530007 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e1240-93f3-4929-a591-b9b4c1ced93e None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:34.605387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88ec6a-321e-4e7f-8b47-297092a90e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:34.607771 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88ec6a-321e-4e7f-8b47-297092a90e42 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:34.608643 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88ec6a-321e-4e7f-8b47-297092a90e42 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:34.609591 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88ec6a-321e-4e7f-8b47-297092a90e42 None admin] PATH_INFO: `/v3/groups/2960f6ca0176451eb54f65211c582146` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:34.612279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88ec6a-321e-4e7f-8b47-297092a90e42 None admin] RBAC: Authorizing `identity:get_group(group_id=2960f6ca0176451eb54f65211c582146)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:34.615702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc88ec6a-321e-4e7f-8b47-297092a90e42 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:34.619510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 177/374] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:34 2022] GET /identity/v3/groups/2960f6ca0176451eb54f65211c582146 => generated 226 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:34.621582 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88ec6a-321e-4e7f-8b47-297092a90e42 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:34.677178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:34.760759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1d929-4170-487d-a04d-91e7a101baf8 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:34.835631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88ec6a-321e-4e7f-8b47-297092a90e42 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:34.840283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3a38a-3b73-487f-90cb-2bceff11a9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:34.841329 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3a38a-3b73-487f-90cb-2bceff11a9e3 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:34.841859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3a38a-3b73-487f-90cb-2bceff11a9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:34.842410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3a38a-3b73-487f-90cb-2bceff11a9e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:34.843699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3a38a-3b73-487f-90cb-2bceff11a9e3 None admin] RBAC: Adding query filter params (group.id=2960f6ca0176451eb54f65211c582146, role.id=eff5cca513be4ed5a9b001c0559fb476, scope.project.id=d0c721e2529541e5966d3c21426e447a) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:34.844046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3a38a-3b73-487f-90cb-2bceff11a9e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:34.848290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3a38a-3b73-487f-90cb-2bceff11a9e3 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:34.856000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:34.862016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 198/375] 10.222.0.94 () {56 vars in 1376 bytes} [Tue Jul 26 14:40:34 2022] GET /identity/v3/role_assignments?group.id=2960f6ca0176451eb54f65211c582146&scope.project.id=d0c721e2529541e5966d3c21426e447a&role.id=eff5cca513be4ed5a9b001c0559fb476 => generated 260 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:34.864108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3a38a-3b73-487f-90cb-2bceff11a9e3 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:34.898934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50d968-3c00-4119-94d2-f121a536adbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:34.899766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50d968-3c00-4119-94d2-f121a536adbb None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:34.900058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50d968-3c00-4119-94d2-f121a536adbb None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:34.900374 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50d968-3c00-4119-94d2-f121a536adbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:35.027713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca50d968-3c00-4119-94d2-f121a536adbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:35.029316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca50d968-3c00-4119-94d2-f121a536adbb None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:35.100043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3a38a-3b73-487f-90cb-2bceff11a9e3 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:35.175012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1eb833-50a4-44d5-9541-85d12a37acfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:35.176773 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1eb833-50a4-44d5-9541-85d12a37acfa None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:35.177254 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1eb833-50a4-44d5-9541-85d12a37acfa None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:35.177728 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1eb833-50a4-44d5-9541-85d12a37acfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:35.179363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1eb833-50a4-44d5-9541-85d12a37acfa None admin] RBAC: Adding query filter params (group.id=2960f6ca0176451eb54f65211c582146, role.id=eff5cca513be4ed5a9b001c0559fb476, scope.project.id=21ad3a8335ad492d858142d787614be0) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:35.179945 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1eb833-50a4-44d5-9541-85d12a37acfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:35.182897 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1eb833-50a4-44d5-9541-85d12a37acfa None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:35.187954 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 178/376] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:35.192050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50d968-3c00-4119-94d2-f121a536adbb None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:35.198784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 199/377] 10.222.0.94 () {56 vars in 1376 bytes} [Tue Jul 26 14:40:34 2022] GET /identity/v3/role_assignments?group.id=2960f6ca0176451eb54f65211c582146&scope.project.id=21ad3a8335ad492d858142d787614be0&role.id=eff5cca513be4ed5a9b001c0559fb476 => generated 260 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:35.211906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:35.355351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50d968-3c00-4119-94d2-f121a536adbb None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:35.440211 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d1bc1-7f96-4fc3-82e0-c94ca51e5f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:35.442462 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d1bc1-7f96-4fc3-82e0-c94ca51e5f15 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:35.443336 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d1bc1-7f96-4fc3-82e0-c94ca51e5f15 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:35.444172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d1bc1-7f96-4fc3-82e0-c94ca51e5f15 None admin] PATH_INFO: `/v3/projects/382829af20d649b9beb375e3c3cc2d50` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:35.446966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d1bc1-7f96-4fc3-82e0-c94ca51e5f15 None admin] RBAC: Authorizing `identity:get_project(project_id=382829af20d649b9beb375e3c3cc2d50)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:35.450439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d1bc1-7f96-4fc3-82e0-c94ca51e5f15 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:35.454315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd366c-ba4e-40f2-be29-3a2b527591c0 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:35.454556 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd366c-ba4e-40f2-be29-3a2b527591c0 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:35.454846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd366c-ba4e-40f2-be29-3a2b527591c0 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:35.455507 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 179/378] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:35 2022] GET /identity/v3/projects/382829af20d649b9beb375e3c3cc2d50 => generated 344 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:35.456364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 200/379] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:35.463699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d1bc1-7f96-4fc3-82e0-c94ca51e5f15 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:35.465133 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44826b7-81ae-4968-9731-db35ef58d30a None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:35.465447 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44826b7-81ae-4968-9731-db35ef58d30a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:35.465733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44826b7-81ae-4968-9731-db35ef58d30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:35.491528 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:35.521182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c44826b7-81ae-4968-9731-db35ef58d30a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:35.526879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c44826b7-81ae-4968-9731-db35ef58d30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:35.738853 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:35.775678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 201/380] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:35.812882 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44826b7-81ae-4968-9731-db35ef58d30a None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:35.819587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d1bc1-7f96-4fc3-82e0-c94ca51e5f15 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:35.875758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4158204-4597-433a-8db3-16a43aa9f3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:35.880659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4158204-4597-433a-8db3-16a43aa9f3e6 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:35.881075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4158204-4597-433a-8db3-16a43aa9f3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:35.881316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4158204-4597-433a-8db3-16a43aa9f3e6 None admin] PATH_INFO: `/v3/groups/0215bf55e19443549c0ca04a4a97823a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:35.889379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4158204-4597-433a-8db3-16a43aa9f3e6 None admin] RBAC: Authorizing `identity:get_group(group_id=0215bf55e19443549c0ca04a4a97823a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:35.890874 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4158204-4597-433a-8db3-16a43aa9f3e6 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:35.893044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 180/381] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:35 2022] GET /identity/v3/groups/0215bf55e19443549c0ca04a4a97823a => generated 229 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:35.902576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4158204-4597-433a-8db3-16a43aa9f3e6 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:36.072445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4158204-4597-433a-8db3-16a43aa9f3e6 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:36.099052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44826b7-81ae-4968-9731-db35ef58d30a None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:36.152168 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f894d0-cf5b-4759-8784-58cd0cb257a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:36.152800 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f894d0-cf5b-4759-8784-58cd0cb257a4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:36.153079 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f894d0-cf5b-4759-8784-58cd0cb257a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:36.153354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f894d0-cf5b-4759-8784-58cd0cb257a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:36.154208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f894d0-cf5b-4759-8784-58cd0cb257a4 None admin] RBAC: Adding query filter params (group.id=0215bf55e19443549c0ca04a4a97823a, role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=382829af20d649b9beb375e3c3cc2d50) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:36.154516 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f894d0-cf5b-4759-8784-58cd0cb257a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:36.156244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f894d0-cf5b-4759-8784-58cd0cb257a4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:36.168848 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 181/382] 10.222.0.94 () {56 vars in 1376 bytes} [Tue Jul 26 14:40:35 2022] GET /identity/v3/role_assignments?group.id=0215bf55e19443549c0ca04a4a97823a&scope.project.id=382829af20d649b9beb375e3c3cc2d50&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 260 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:36.181001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dccac-5c88-4be3-a9a8-086c13d1a86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:36.182311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dccac-5c88-4be3-a9a8-086c13d1a86c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:36.182691 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dccac-5c88-4be3-a9a8-086c13d1a86c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:36.183095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dccac-5c88-4be3-a9a8-086c13d1a86c None admin] PATH_INFO: `/v3/roles/member` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:36.197771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63dccac-5c88-4be3-a9a8-086c13d1a86c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-f63dccac-5c88-4be3-a9a8-086c13d1a86c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:36.204086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:36.206252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:36.208774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:36.208774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:36.208774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:36.208774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:36.208774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:36.208774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:36.208774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:36.208774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:36.208774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:36.208774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:36.208774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:36.208774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:36.208774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 202/383] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:36.208774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:36.219937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f894d0-cf5b-4759-8784-58cd0cb257a4 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:36.240948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:36.463151 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f894d0-cf5b-4759-8784-58cd0cb257a4 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:36.528119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644dcbc-ecfa-4140-a332-05e1dfc094e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:36.529217 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644dcbc-ecfa-4140-a332-05e1dfc094e3 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:36.529710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644dcbc-ecfa-4140-a332-05e1dfc094e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:36.530200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644dcbc-ecfa-4140-a332-05e1dfc094e3 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:36.531106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644dcbc-ecfa-4140-a332-05e1dfc094e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:36.531393 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644dcbc-ecfa-4140-a332-05e1dfc094e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:36.532859 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644dcbc-ecfa-4140-a332-05e1dfc094e3 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:36.551433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 182/384] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:36.559576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dccac-5c88-4be3-a9a8-086c13d1a86c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:36.713625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dccac-5c88-4be3-a9a8-086c13d1a86c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:36.787031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d0f99-a355-41b8-a1ca-991bf0d473c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:36.788390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d0f99-a355-41b8-a1ca-991bf0d473c2 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:36.788873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d0f99-a355-41b8-a1ca-991bf0d473c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:36.789314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d0f99-a355-41b8-a1ca-991bf0d473c2 None admin] PATH_INFO: `/v3/groups/2960f6ca0176451eb54f65211c582146` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:36.791419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d0f99-a355-41b8-a1ca-991bf0d473c2 None admin] RBAC: Authorizing `identity:get_group(group_id=2960f6ca0176451eb54f65211c582146)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:36.794315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d0f99-a355-41b8-a1ca-991bf0d473c2 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:36.803762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 203/385] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:36 2022] GET /identity/v3/groups/2960f6ca0176451eb54f65211c582146 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:36.806895 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644dcbc-ecfa-4140-a332-05e1dfc094e3 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:36.980657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:37.057780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644dcbc-ecfa-4140-a332-05e1dfc094e3 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:37.129093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a43eb-efa1-49a9-82c5-1e8d8ed1e7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:37.129774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a43eb-efa1-49a9-82c5-1e8d8ed1e7d0 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:37.130065 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a43eb-efa1-49a9-82c5-1e8d8ed1e7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:37.130348 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a43eb-efa1-49a9-82c5-1e8d8ed1e7d0 None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:37.136535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a43eb-efa1-49a9-82c5-1e8d8ed1e7d0 None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:37.138290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a43eb-efa1-49a9-82c5-1e8d8ed1e7d0 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:37.140723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 183/386] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:36 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:37.149080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d0f99-a355-41b8-a1ca-991bf0d473c2 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:37.328129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:37.399039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d0f99-a355-41b8-a1ca-991bf0d473c2 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:37.458903 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f87426-a84e-4c84-814c-e1d72602c030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:37.459663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f87426-a84e-4c84-814c-e1d72602c030 None admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:37.460000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f87426-a84e-4c84-814c-e1d72602c030 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:37.460285 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f87426-a84e-4c84-814c-e1d72602c030 None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a/groups/2960f6ca0176451eb54f65211c582146/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:37.462588 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f87426-a84e-4c84-814c-e1d72602c030 None admin] RBAC: Authorizing `identity:create_grant(project_id=d0c721e2529541e5966d3c21426e447a, group_id=2960f6ca0176451eb54f65211c582146, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:37.464247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f87426-a84e-4c84-814c-e1d72602c030 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:37.564890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 204/387] 10.222.0.94 () {58 vars in 1442 bytes} [Tue Jul 26 14:40:37 2022] PUT /identity/v3/projects/d0c721e2529541e5966d3c21426e447a/groups/2960f6ca0176451eb54f65211c582146/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:40:37.934453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d87cc-4362-4b9f-92d0-27c851b2a4f5 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:37.935000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d87cc-4362-4b9f-92d0-27c851b2a4f5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:37.935378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d87cc-4362-4b9f-92d0-27c851b2a4f5 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:37.938657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 184/388] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:37.948470 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafbd68-d9d7-4ef5-bcd3-bf012ae22c9f None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:37.948992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafbd68-d9d7-4ef5-bcd3-bf012ae22c9f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:37.949433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafbd68-d9d7-4ef5-bcd3-bf012ae22c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:37.990669 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-5fafbd68-d9d7-4ef5-bcd3-bf012ae22c9f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:37.992883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5fafbd68-d9d7-4ef5-bcd3-bf012ae22c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:38.170522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 205/389] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:38.250818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d87cc-4362-4b9f-92d0-27c851b2a4f5 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:38.449754 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d87cc-4362-4b9f-92d0-27c851b2a4f5 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:38.508500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea30e9f-f4f7-4776-aaea-6bf4d00e8ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:38.509221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea30e9f-f4f7-4776-aaea-6bf4d00e8ebd None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:38.509530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea30e9f-f4f7-4776-aaea-6bf4d00e8ebd None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:38.509856 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea30e9f-f4f7-4776-aaea-6bf4d00e8ebd None admin] PATH_INFO: `/v3/roles/member` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:38.514266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:38.531879 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea30e9f-f4f7-4776-aaea-6bf4d00e8ebd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-5ea30e9f-f4f7-4776-aaea-6bf4d00e8ebd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:38.550031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:38.552477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:38.555311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:38.555311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:38.555311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:38.555311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:38.555311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:38.555311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:38.555311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:38.555311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:38.555311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:38.555311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:38.555311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:38.555311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:38.555311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 185/390] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:38.557890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafbd68-d9d7-4ef5-bcd3-bf012ae22c9f None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:38.702388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:38.835915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafbd68-d9d7-4ef5-bcd3-bf012ae22c9f None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:38.844586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a6c6b-2fe0-46df-9574-c24fb84d7835 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:38.845037 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a6c6b-2fe0-46df-9574-c24fb84d7835 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:38.845189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a6c6b-2fe0-46df-9574-c24fb84d7835 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:38.849328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 186/391] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:38.855179 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a1245-bef3-4f18-9adb-4732fbf967e0 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:38.859688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a1245-bef3-4f18-9adb-4732fbf967e0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:38.859688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a1245-bef3-4f18-9adb-4732fbf967e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:38.879718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-bb0a1245-bef3-4f18-9adb-4732fbf967e0 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:38.881248 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-bb0a1245-bef3-4f18-9adb-4732fbf967e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:38.938411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e8cbc-ee2a-4fd5-b56a-e27fc8523fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:38.939137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e8cbc-ee2a-4fd5-b56a-e27fc8523fb0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:38.939423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e8cbc-ee2a-4fd5-b56a-e27fc8523fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:38.939723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e8cbc-ee2a-4fd5-b56a-e27fc8523fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:38.940519 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e8cbc-ee2a-4fd5-b56a-e27fc8523fb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:38.940944 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e8cbc-ee2a-4fd5-b56a-e27fc8523fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:38.942647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e8cbc-ee2a-4fd5-b56a-e27fc8523fb0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:38.955070 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 206/392] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:38.963970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e8cbc-ee2a-4fd5-b56a-e27fc8523fb0 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:39.103980 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 187/393] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:40:39.156649 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e8cbc-ee2a-4fd5-b56a-e27fc8523fb0 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:39.176097 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:39.187505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a1245-bef3-4f18-9adb-4732fbf967e0 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:39.212836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd06e3f-3a01-4ec8-8d51-26b7b6ef4cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:39.214106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd06e3f-3a01-4ec8-8d51-26b7b6ef4cdc None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:39.219692 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd06e3f-3a01-4ec8-8d51-26b7b6ef4cdc None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:39.219692 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd06e3f-3a01-4ec8-8d51-26b7b6ef4cdc None admin] PATH_INFO: `/v3/groups/2960f6ca0176451eb54f65211c582146` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:39.219692 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd06e3f-3a01-4ec8-8d51-26b7b6ef4cdc None admin] RBAC: Authorizing `identity:get_group(group_id=2960f6ca0176451eb54f65211c582146)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:39.220178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd06e3f-3a01-4ec8-8d51-26b7b6ef4cdc None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:39.224598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 207/394] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:38 2022] GET /identity/v3/groups/2960f6ca0176451eb54f65211c582146 => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:39.234387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd06e3f-3a01-4ec8-8d51-26b7b6ef4cdc None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:39.434604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd06e3f-3a01-4ec8-8d51-26b7b6ef4cdc None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:39.471906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a1245-bef3-4f18-9adb-4732fbf967e0 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:39.509264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:39.522800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c92e82-1fcc-47a6-96a9-43a08e384023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:39.522800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c92e82-1fcc-47a6-96a9-43a08e384023 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:39.522800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c92e82-1fcc-47a6-96a9-43a08e384023 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:39.522800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c92e82-1fcc-47a6-96a9-43a08e384023 None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:39.525571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c92e82-1fcc-47a6-96a9-43a08e384023 None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:39.528216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c92e82-1fcc-47a6-96a9-43a08e384023 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:39.531620 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a32154-5f73-4f1f-86f5-8c8a4c6222d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:39.532112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 208/395] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:39 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:39.533788 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a32154-5f73-4f1f-86f5-8c8a4c6222d3 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:39.534593 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846f4dd-1c87-427b-8b68-32d51a0cec4d None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:39.534917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a32154-5f73-4f1f-86f5-8c8a4c6222d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:39.535098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846f4dd-1c87-427b-8b68-32d51a0cec4d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:39.535760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846f4dd-1c87-427b-8b68-32d51a0cec4d None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:39.536281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a32154-5f73-4f1f-86f5-8c8a4c6222d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:39.543519 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 209/396] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:39.543519 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846f4dd-1c87-427b-8b68-32d51a0cec4d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:39.566651 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a32154-5f73-4f1f-86f5-8c8a4c6222d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-90a32154-5f73-4f1f-86f5-8c8a4c6222d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:39.578599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:39.580531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:39.583318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:39.583318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:39.583318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:39.583318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:39.583318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:39.583318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:39.583318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:39.583318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:39.583318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:39.583318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:39.583318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:39.583318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:39.583318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 188/397] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:40:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:39.587388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748f6c5-e88f-4d7b-9372-0d8d2b2f3ddd None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:39.587671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748f6c5-e88f-4d7b-9372-0d8d2b2f3ddd None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:39.591866 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748f6c5-e88f-4d7b-9372-0d8d2b2f3ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:39.616741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-1748f6c5-e88f-4d7b-9372-0d8d2b2f3ddd None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:39.618208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1748f6c5-e88f-4d7b-9372-0d8d2b2f3ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:39.655847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:39.715579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 189/398] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:39.715579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748f6c5-e88f-4d7b-9372-0d8d2b2f3ddd None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:39.747082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846f4dd-1c87-427b-8b68-32d51a0cec4d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:39.799918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18e333-e519-49f0-9af3-77818703368f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:39.801722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18e333-e519-49f0-9af3-77818703368f None admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:39.802143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18e333-e519-49f0-9af3-77818703368f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:39.802432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18e333-e519-49f0-9af3-77818703368f None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0/groups/2960f6ca0176451eb54f65211c582146/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:39.805298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18e333-e519-49f0-9af3-77818703368f None admin] RBAC: Authorizing `identity:create_grant(project_id=21ad3a8335ad492d858142d787614be0, group_id=2960f6ca0176451eb54f65211c582146, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:39.808135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18e333-e519-49f0-9af3-77818703368f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:39.847139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748f6c5-e88f-4d7b-9372-0d8d2b2f3ddd None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:39.871614 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:39.899161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fbfc2-0862-49ac-8175-5d160ef97d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:39.900304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fbfc2-0862-49ac-8175-5d160ef97d42 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:39.900886 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fbfc2-0862-49ac-8175-5d160ef97d42 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:39.901481 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fbfc2-0862-49ac-8175-5d160ef97d42 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:39.902530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fbfc2-0862-49ac-8175-5d160ef97d42 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:39.903094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fbfc2-0862-49ac-8175-5d160ef97d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:39.904942 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fbfc2-0862-49ac-8175-5d160ef97d42 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:39.907775 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 210/399] 10.222.0.94 () {58 vars in 1442 bytes} [Tue Jul 26 14:40:39 2022] PUT /identity/v3/projects/21ad3a8335ad492d858142d787614be0/groups/2960f6ca0176451eb54f65211c582146/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:40:39.909373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18e333-e519-49f0-9af3-77818703368f None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:39.932497 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 190/400] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:40:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:39.939247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fbfc2-0862-49ac-8175-5d160ef97d42 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:40.173350 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18e333-e519-49f0-9af3-77818703368f None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:40.203914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:40.204545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fbfc2-0862-49ac-8175-5d160ef97d42 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:40.236289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3fabf-e5d0-49f9-ac61-fa13ca4f9616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:40.237348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3fabf-e5d0-49f9-ac61-fa13ca4f9616 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:40.238543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3fabf-e5d0-49f9-ac61-fa13ca4f9616 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:40.238874 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3fabf-e5d0-49f9-ac61-fa13ca4f9616 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:40.262986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3fabf-e5d0-49f9-ac61-fa13ca4f9616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-cfc3fabf-e5d0-49f9-ac61-fa13ca4f9616 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:40.271144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:40.273302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:40.275009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:40.275009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:40.275009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:40.275009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:40.275009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:40.275009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:40.275009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:40.275009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:40.275009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:40.275009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:40.275009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:40.275009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:40.275009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 211/401] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:40:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:40.280407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f7e68-9f6c-4b3f-9061-b556993dffcf None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:40.281245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f7e68-9f6c-4b3f-9061-b556993dffcf None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:40.282039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f7e68-9f6c-4b3f-9061-b556993dffcf None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:40.283272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc69e97-50cb-4af0-8d1c-6ec9753b2534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:40.284363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc69e97-50cb-4af0-8d1c-6ec9753b2534 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:40.285527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc69e97-50cb-4af0-8d1c-6ec9753b2534 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:40.286068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc69e97-50cb-4af0-8d1c-6ec9753b2534 None admin] PATH_INFO: `/v3/groups/2960f6ca0176451eb54f65211c582146` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:40.291355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 212/402] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:40 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:40.291355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f7e68-9f6c-4b3f-9061-b556993dffcf None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:40.295146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc69e97-50cb-4af0-8d1c-6ec9753b2534 None admin] RBAC: Authorizing `identity:get_group(group_id=2960f6ca0176451eb54f65211c582146)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:40.296896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc69e97-50cb-4af0-8d1c-6ec9753b2534 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:40.299156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 191/403] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:39 2022] GET /identity/v3/groups/2960f6ca0176451eb54f65211c582146 => generated 226 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:40.300667 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71ed3-3edd-4118-b14c-1d1385dec6c8 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:40.301102 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71ed3-3edd-4118-b14c-1d1385dec6c8 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:40.302061 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe71ed3-3edd-4118-b14c-1d1385dec6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:40.340958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-fbe71ed3-3edd-4118-b14c-1d1385dec6c8 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:40.343316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-fbe71ed3-3edd-4118-b14c-1d1385dec6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:40.443716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:40.485450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f7e68-9f6c-4b3f-9061-b556993dffcf None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:40.512119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 192/404] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:40.514197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe71ed3-3edd-4118-b14c-1d1385dec6c8 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:40.531497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71688cef-254f-458c-be0c-2bc5daf2d8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:40.532237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71688cef-254f-458c-be0c-2bc5daf2d8fb None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:40.532520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71688cef-254f-458c-be0c-2bc5daf2d8fb None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:40.532791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71688cef-254f-458c-be0c-2bc5daf2d8fb None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:40.533537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71688cef-254f-458c-be0c-2bc5daf2d8fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:40.533917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71688cef-254f-458c-be0c-2bc5daf2d8fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:40.536241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71688cef-254f-458c-be0c-2bc5daf2d8fb None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:40.548765 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 213/405] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:40:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:40.560899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71688cef-254f-458c-be0c-2bc5daf2d8fb None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:40.612889 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:40.709159 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71688cef-254f-458c-be0c-2bc5daf2d8fb None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:40.752550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885fe18-b2b6-4b03-a72d-393014925b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:40.753168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885fe18-b2b6-4b03-a72d-393014925b76 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:40.754204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885fe18-b2b6-4b03-a72d-393014925b76 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:40.754584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885fe18-b2b6-4b03-a72d-393014925b76 None admin] PATH_INFO: `/v3/groups/2960f6ca0176451eb54f65211c582146` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:40.756020 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885fe18-b2b6-4b03-a72d-393014925b76 None admin] RBAC: Authorizing `identity:get_group(group_id=2960f6ca0176451eb54f65211c582146)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:40.757616 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885fe18-b2b6-4b03-a72d-393014925b76 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:40.764572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 214/406] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:40 2022] GET /identity/v3/groups/2960f6ca0176451eb54f65211c582146 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:40.765611 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885fe18-b2b6-4b03-a72d-393014925b76 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:40.766277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe71ed3-3edd-4118-b14c-1d1385dec6c8 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:40.842902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781c1a9-deca-4e1a-914d-d4d289ecd03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:40.844920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781c1a9-deca-4e1a-914d-d4d289ecd03e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:40.845733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781c1a9-deca-4e1a-914d-d4d289ecd03e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:40.846646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781c1a9-deca-4e1a-914d-d4d289ecd03e None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:40.850175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781c1a9-deca-4e1a-914d-d4d289ecd03e None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:40.853907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781c1a9-deca-4e1a-914d-d4d289ecd03e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:40.858433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 193/407] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:40 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:40.860679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781c1a9-deca-4e1a-914d-d4d289ecd03e None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:40.906757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:40.910678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885fe18-b2b6-4b03-a72d-393014925b76 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:40.959769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085ba83-fc51-4d1b-88fd-6db79e31176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:40.960859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085ba83-fc51-4d1b-88fd-6db79e31176c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:40.961458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085ba83-fc51-4d1b-88fd-6db79e31176c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:40.962042 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085ba83-fc51-4d1b-88fd-6db79e31176c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:40.969087 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:40.978552 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085ba83-fc51-4d1b-88fd-6db79e31176c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-9085ba83-fc51-4d1b-88fd-6db79e31176c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:40.985628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:40.988761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:40.990320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:40.990320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:40.990320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:40.990320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:40.990320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:40.990320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:40.990320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:40.990320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:40.990320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:40.990320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:40.990320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:40:40.990320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:40.992984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 215/408] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:40:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:40.994128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085ba83-fc51-4d1b-88fd-6db79e31176c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:41.098919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781c1a9-deca-4e1a-914d-d4d289ecd03e None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:41.157061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085ba83-fc51-4d1b-88fd-6db79e31176c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:41.161792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfacc4-90d8-4ca0-bd45-070797aea111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:41.163172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfacc4-90d8-4ca0-bd45-070797aea111 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:41.163702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfacc4-90d8-4ca0-bd45-070797aea111 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:41.164140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfacc4-90d8-4ca0-bd45-070797aea111 None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:41.166269 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cfacc4-90d8-4ca0-bd45-070797aea111 None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:41.168846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cfacc4-90d8-4ca0-bd45-070797aea111 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:41.177784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 194/409] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:40 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:41.177784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfacc4-90d8-4ca0-bd45-070797aea111 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:41.224034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f3524-8f17-4efb-816c-724ea68990b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:41.228974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f3524-8f17-4efb-816c-724ea68990b3 None admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:41.228974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f3524-8f17-4efb-816c-724ea68990b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:41.228974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f3524-8f17-4efb-816c-724ea68990b3 None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a/groups/2960f6ca0176451eb54f65211c582146/roles/eff5cca513be4ed5a9b001c0559fb476` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:41.230467 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f3524-8f17-4efb-816c-724ea68990b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d0c721e2529541e5966d3c21426e447a, group_id=2960f6ca0176451eb54f65211c582146, role_id=eff5cca513be4ed5a9b001c0559fb476)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:41.233713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f3524-8f17-4efb-816c-724ea68990b3 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:41.370445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfacc4-90d8-4ca0-bd45-070797aea111 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:41.372524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 216/410] 10.222.0.94 () {58 vars in 1442 bytes} [Tue Jul 26 14:40:40 2022] PUT /identity/v3/projects/d0c721e2529541e5966d3c21426e447a/groups/2960f6ca0176451eb54f65211c582146/roles/eff5cca513be4ed5a9b001c0559fb476 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:40:41.374050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5374699-6c64-4b84-baf2-dfe512a9191b None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:41.374465 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5374699-6c64-4b84-baf2-dfe512a9191b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:41.374876 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5374699-6c64-4b84-baf2-dfe512a9191b None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:41.377387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 217/411] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:41.378436 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5374699-6c64-4b84-baf2-dfe512a9191b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:41.417948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:41.448594 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:41.470602 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee411d8-8a86-49a0-9471-fa576a2ee742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:41.471653 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee411d8-8a86-49a0-9471-fa576a2ee742 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:41.473119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee411d8-8a86-49a0-9471-fa576a2ee742 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:41.473742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee411d8-8a86-49a0-9471-fa576a2ee742 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:41.475412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee411d8-8a86-49a0-9471-fa576a2ee742 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:41.476257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee411d8-8a86-49a0-9471-fa576a2ee742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:41.479181 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee411d8-8a86-49a0-9471-fa576a2ee742 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:41.508553 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 195/412] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:41.511302 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d3529-e19b-4637-8797-1a5d86f99cd0 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:41.511865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d3529-e19b-4637-8797-1a5d86f99cd0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:41.512395 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d3529-e19b-4637-8797-1a5d86f99cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:41.554922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-7c4d3529-e19b-4637-8797-1a5d86f99cd0 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:41.558382 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-7c4d3529-e19b-4637-8797-1a5d86f99cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:41.624714 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5374699-6c64-4b84-baf2-dfe512a9191b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:41.706988 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90166e68-09e6-4967-8c37-47a5a44b4ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:41.708220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90166e68-09e6-4967-8c37-47a5a44b4ee5 None admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:41.709349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90166e68-09e6-4967-8c37-47a5a44b4ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:41.709889 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90166e68-09e6-4967-8c37-47a5a44b4ee5 None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0/groups/2960f6ca0176451eb54f65211c582146/roles/eff5cca513be4ed5a9b001c0559fb476` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:41.716766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90166e68-09e6-4967-8c37-47a5a44b4ee5 None admin] RBAC: Authorizing `identity:create_grant(project_id=21ad3a8335ad492d858142d787614be0, group_id=2960f6ca0176451eb54f65211c582146, role_id=eff5cca513be4ed5a9b001c0559fb476)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:41.720000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90166e68-09e6-4967-8c37-47a5a44b4ee5 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:41.725045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 196/413] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:41.726413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d3529-e19b-4637-8797-1a5d86f99cd0 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:41.822955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 218/414] 10.222.0.94 () {58 vars in 1442 bytes} [Tue Jul 26 14:40:41 2022] PUT /identity/v3/projects/21ad3a8335ad492d858142d787614be0/groups/2960f6ca0176451eb54f65211c582146/roles/eff5cca513be4ed5a9b001c0559fb476 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:40:41.834190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90166e68-09e6-4967-8c37-47a5a44b4ee5 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:41.908380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d3529-e19b-4637-8797-1a5d86f99cd0 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:41.912851 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:41.974092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f876b-8189-4b12-8b0a-b56175b09f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:41.975513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f876b-8189-4b12-8b0a-b56175b09f32 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:41.975984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f876b-8189-4b12-8b0a-b56175b09f32 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:41.976509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f876b-8189-4b12-8b0a-b56175b09f32 None admin] PATH_INFO: `/v3/groups/0215bf55e19443549c0ca04a4a97823a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:41.979248 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f876b-8189-4b12-8b0a-b56175b09f32 None admin] RBAC: Authorizing `identity:get_group(group_id=0215bf55e19443549c0ca04a4a97823a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:41.983500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f876b-8189-4b12-8b0a-b56175b09f32 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:41.988676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 197/415] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:41 2022] GET /identity/v3/groups/0215bf55e19443549c0ca04a4a97823a => generated 229 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:41.998585 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f876b-8189-4b12-8b0a-b56175b09f32 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:42.100645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:42.153600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90166e68-09e6-4967-8c37-47a5a44b4ee5 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:42.233323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70308a87-deac-4242-af4e-df192318d337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:42.235517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70308a87-deac-4242-af4e-df192318d337 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:42.236315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70308a87-deac-4242-af4e-df192318d337 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:42.237007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70308a87-deac-4242-af4e-df192318d337 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:42.262408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70308a87-deac-4242-af4e-df192318d337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-70308a87-deac-4242-af4e-df192318d337 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:42.275452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:42.276705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:42.277713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:42.277713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:42.277713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:42.277713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:42.277713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:42.277713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:42.277713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:42.277713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:42.277713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:42.277713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:42.277713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:42.277713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:42.277713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 219/416] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 442 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:42.286625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70308a87-deac-4242-af4e-df192318d337 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:42.318984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f876b-8189-4b12-8b0a-b56175b09f32 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:42.418207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e90c4d-0bb1-4f64-9207-d5d2146c1b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:42.419244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90c4d-0bb1-4f64-9207-d5d2146c1b7a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:42.419740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90c4d-0bb1-4f64-9207-d5d2146c1b7a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:42.420186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90c4d-0bb1-4f64-9207-d5d2146c1b7a None admin] PATH_INFO: `/v3/projects/382829af20d649b9beb375e3c3cc2d50` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:42.424876 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e90c4d-0bb1-4f64-9207-d5d2146c1b7a None admin] RBAC: Authorizing `identity:get_project(project_id=382829af20d649b9beb375e3c3cc2d50)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:42.428990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e90c4d-0bb1-4f64-9207-d5d2146c1b7a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:42.437140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 198/417] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:41 2022] GET /identity/v3/projects/382829af20d649b9beb375e3c3cc2d50 => generated 344 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:42.446960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e90c4d-0bb1-4f64-9207-d5d2146c1b7a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:42.509932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70308a87-deac-4242-af4e-df192318d337 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:42.571880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:42.592414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde6a54-a435-4bd8-a8ce-4a49635df4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:42.594120 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde6a54-a435-4bd8-a8ce-4a49635df4f3 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:42.594827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde6a54-a435-4bd8-a8ce-4a49635df4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:42.595341 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde6a54-a435-4bd8-a8ce-4a49635df4f3 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:42.596623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde6a54-a435-4bd8-a8ce-4a49635df4f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:42.597146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde6a54-a435-4bd8-a8ce-4a49635df4f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:42.599486 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde6a54-a435-4bd8-a8ce-4a49635df4f3 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:42.617005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 220/418] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:42.625784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde6a54-a435-4bd8-a8ce-4a49635df4f3 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:42.631252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:42.811081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e90c4d-0bb1-4f64-9207-d5d2146c1b7a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:42.835015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde6a54-a435-4bd8-a8ce-4a49635df4f3 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:42.884285 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f456346-5db4-4fc9-9433-8c40afa21594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:42.884978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f456346-5db4-4fc9-9433-8c40afa21594 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:42.885585 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f456346-5db4-4fc9-9433-8c40afa21594 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:42.885889 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f456346-5db4-4fc9-9433-8c40afa21594 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:42.983099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b33c7-23e9-443e-8d03-8f67ffdd819a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:42.984431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b33c7-23e9-443e-8d03-8f67ffdd819a None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:42.985015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b33c7-23e9-443e-8d03-8f67ffdd819a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:42.985610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b33c7-23e9-443e-8d03-8f67ffdd819a None admin] PATH_INFO: `/v3/projects/382829af20d649b9beb375e3c3cc2d50/groups/0215bf55e19443549c0ca04a4a97823a/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:42.988871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b33c7-23e9-443e-8d03-8f67ffdd819a None admin] RBAC: Authorizing `identity:create_grant(project_id=382829af20d649b9beb375e3c3cc2d50, group_id=0215bf55e19443549c0ca04a4a97823a, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:42.990982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b33c7-23e9-443e-8d03-8f67ffdd819a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:42.997096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f456346-5db4-4fc9-9433-8c40afa21594 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:43.000164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f456346-5db4-4fc9-9433-8c40afa21594 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:43.003958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:43.090000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 199/419] 10.222.0.94 () {58 vars in 1442 bytes} [Tue Jul 26 14:40:42 2022] PUT /identity/v3/projects/382829af20d649b9beb375e3c3cc2d50/groups/0215bf55e19443549c0ca04a4a97823a/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:40:43.092627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9f482-f3b1-41dd-937b-d49b52104bfb None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:43.093571 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9f482-f3b1-41dd-937b-d49b52104bfb None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:43.094461 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9f482-f3b1-41dd-937b-d49b52104bfb None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:43.104246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 200/420] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:43 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:43.109200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b1a6d-0b01-4027-bb81-db2ed9f695dc None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:43.109981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b1a6d-0b01-4027-bb81-db2ed9f695dc None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:43.110627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b1a6d-0b01-4027-bb81-db2ed9f695dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:43.158958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-415b1a6d-0b01-4027-bb81-db2ed9f695dc None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:43.163400 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-415b1a6d-0b01-4027-bb81-db2ed9f695dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:43.286535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 221/421] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 14:40:43.298414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f456346-5db4-4fc9-9433-8c40afa21594 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:43.382337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 201/422] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:43.421219 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b1a6d-0b01-4027-bb81-db2ed9f695dc None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:43.492069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:43.506451 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f456346-5db4-4fc9-9433-8c40afa21594 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:43.508737 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:43.601095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216f459-9060-4e5a-8c83-0f2231ebbdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:43.601815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216f459-9060-4e5a-8c83-0f2231ebbdd5 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:43.602115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216f459-9060-4e5a-8c83-0f2231ebbdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:43.602354 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216f459-9060-4e5a-8c83-0f2231ebbdd5 None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:43.605552 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216f459-9060-4e5a-8c83-0f2231ebbdd5 None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:43.607192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216f459-9060-4e5a-8c83-0f2231ebbdd5 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:43.628554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 222/423] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:43 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:43.628554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216f459-9060-4e5a-8c83-0f2231ebbdd5 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:43.646479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b1a6d-0b01-4027-bb81-db2ed9f695dc None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:43.756995 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1837942-6d79-4a6b-ad1a-564e4a932733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:43.757681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1837942-6d79-4a6b-ad1a-564e4a932733 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:43.757969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1837942-6d79-4a6b-ad1a-564e4a932733 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:43.758250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1837942-6d79-4a6b-ad1a-564e4a932733 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:43.785324 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1837942-6d79-4a6b-ad1a-564e4a932733 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-a1837942-6d79-4a6b-ad1a-564e4a932733 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:43.796244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:43.797250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:43.798164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:43.798164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:43.798164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:43.798164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:43.798164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:43.798164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:43.798164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:43.798164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:43.798164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:43.798164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:43.798164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 14:40:43.798164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:43.801489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 202/424] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:40:43.806757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1837942-6d79-4a6b-ad1a-564e4a932733 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:43.950230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216f459-9060-4e5a-8c83-0f2231ebbdd5 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:44.026219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d09d9-b83b-4d70-b95f-565016ff12a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:44.027344 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d09d9-b83b-4d70-b95f-565016ff12a0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:44.027715 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d09d9-b83b-4d70-b95f-565016ff12a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:44.027989 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d09d9-b83b-4d70-b95f-565016ff12a0 None admin] PATH_INFO: `/v3/groups/2960f6ca0176451eb54f65211c582146` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:44.029577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d09d9-b83b-4d70-b95f-565016ff12a0 None admin] RBAC: Authorizing `identity:get_group(group_id=2960f6ca0176451eb54f65211c582146)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:44.031553 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d09d9-b83b-4d70-b95f-565016ff12a0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:44.034653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 223/425] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:43 2022] GET /identity/v3/groups/2960f6ca0176451eb54f65211c582146 => generated 226 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:44.042201 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d09d9-b83b-4d70-b95f-565016ff12a0 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:44.103972 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:44.117318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1837942-6d79-4a6b-ad1a-564e4a932733 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:44.175507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:44.184604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35467f0-5f80-40b6-a335-47ae1f08ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:44.185262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35467f0-5f80-40b6-a335-47ae1f08ee40 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:44.185597 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35467f0-5f80-40b6-a335-47ae1f08ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:44.186014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35467f0-5f80-40b6-a335-47ae1f08ee40 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:44.186909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35467f0-5f80-40b6-a335-47ae1f08ee40 None admin] RBAC: Adding query filter params (name=member) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:44.187342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35467f0-5f80-40b6-a335-47ae1f08ee40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:44.193994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35467f0-5f80-40b6-a335-47ae1f08ee40 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:44.220404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 203/426] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:40:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:44.229167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35467f0-5f80-40b6-a335-47ae1f08ee40 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:44.356565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d09d9-b83b-4d70-b95f-565016ff12a0 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:44.468700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057518f-8a5b-4c86-bf13-1cbdd6feccb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:44.469764 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057518f-8a5b-4c86-bf13-1cbdd6feccb9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:44.470247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057518f-8a5b-4c86-bf13-1cbdd6feccb9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:44.470785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057518f-8a5b-4c86-bf13-1cbdd6feccb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:44.472755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057518f-8a5b-4c86-bf13-1cbdd6feccb9 None admin] RBAC: Adding query filter params (group.id=2960f6ca0176451eb54f65211c582146, role.id=6a924f1d619643b3be92bab6fddb3eb2, scope.project.id=d0c721e2529541e5966d3c21426e447a) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:44.473302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057518f-8a5b-4c86-bf13-1cbdd6feccb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:44.475328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057518f-8a5b-4c86-bf13-1cbdd6feccb9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:44.501801 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 224/427] 10.222.0.94 () {56 vars in 1376 bytes} [Tue Jul 26 14:40:44 2022] GET /identity/v3/role_assignments?group.id=2960f6ca0176451eb54f65211c582146&scope.project.id=d0c721e2529541e5966d3c21426e447a&role.id=6a924f1d619643b3be92bab6fddb3eb2 => generated 611 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:44.540770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35467f0-5f80-40b6-a335-47ae1f08ee40 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:44.619720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7a2cd-d201-4732-801b-d2b37f3b7147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:44.620406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7a2cd-d201-4732-801b-d2b37f3b7147 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:44.620799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7a2cd-d201-4732-801b-d2b37f3b7147 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:44.621111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7a2cd-d201-4732-801b-d2b37f3b7147 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:44.684621 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:44.819681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7a2cd-d201-4732-801b-d2b37f3b7147 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:44.822704 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7a2cd-d201-4732-801b-d2b37f3b7147 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:44.910483 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586cf23-c09a-484a-9d03-be68a773b99a None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:44.912079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586cf23-c09a-484a-9d03-be68a773b99a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:44.912761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586cf23-c09a-484a-9d03-be68a773b99a None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:44.916364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 225/428] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:44.922147 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f808ace-3f2d-4efb-bd34-afeb521237c7 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:44.922776 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f808ace-3f2d-4efb-bd34-afeb521237c7 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:44.923256 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f808ace-3f2d-4efb-bd34-afeb521237c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:44.969030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-5f808ace-3f2d-4efb-bd34-afeb521237c7 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:44.970817 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5f808ace-3f2d-4efb-bd34-afeb521237c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:45.073531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 204/429] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 847 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:45.085191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7a2cd-d201-4732-801b-d2b37f3b7147 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:45.169413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 226/430] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:40:45.205441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:45.213470 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f808ace-3f2d-4efb-bd34-afeb521237c7 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:45.291523 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:45.327574 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7a2cd-d201-4732-801b-d2b37f3b7147 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:45.410064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f55f77-6b67-4b8b-a2ae-8c59da385934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:45.412211 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f55f77-6b67-4b8b-a2ae-8c59da385934 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:45.413411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f55f77-6b67-4b8b-a2ae-8c59da385934 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:45.414459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f55f77-6b67-4b8b-a2ae-8c59da385934 None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:45.417184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f55f77-6b67-4b8b-a2ae-8c59da385934 None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:45.420457 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f55f77-6b67-4b8b-a2ae-8c59da385934 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:45.429873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 205/431] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:45 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:45.437557 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f55f77-6b67-4b8b-a2ae-8c59da385934 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:45.451543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f808ace-3f2d-4efb-bd34-afeb521237c7 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:45.502398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589af3a-6507-4d28-b7e9-cc99e3b7e2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:45.503337 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589af3a-6507-4d28-b7e9-cc99e3b7e2b4 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:45.503728 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589af3a-6507-4d28-b7e9-cc99e3b7e2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:45.504310 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589af3a-6507-4d28-b7e9-cc99e3b7e2b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:45.522832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b589af3a-6507-4d28-b7e9-cc99e3b7e2b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-b589af3a-6507-4d28-b7e9-cc99e3b7e2b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:45.531143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:45.532686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:45.534012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:45.534012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:45.534012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:45.534012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:45.534012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:45.534012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:45.534012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:45.534012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:45.534012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:45.534012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:45.534012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:45.534012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:45.539321 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 227/432] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:40:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:40:45.549049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589af3a-6507-4d28-b7e9-cc99e3b7e2b4 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:45.657024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f55f77-6b67-4b8b-a2ae-8c59da385934 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:45.703372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:45.730541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf94531-55c7-4359-b554-5c02181d12a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:45.731585 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf94531-55c7-4359-b554-5c02181d12a4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:45.732003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf94531-55c7-4359-b554-5c02181d12a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:45.732474 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf94531-55c7-4359-b554-5c02181d12a4 None admin] PATH_INFO: `/v3/groups/2960f6ca0176451eb54f65211c582146` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:45.734385 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf94531-55c7-4359-b554-5c02181d12a4 None admin] RBAC: Authorizing `identity:get_group(group_id=2960f6ca0176451eb54f65211c582146)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:45.736807 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf94531-55c7-4359-b554-5c02181d12a4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:45.743752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 206/433] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:45 2022] GET /identity/v3/groups/2960f6ca0176451eb54f65211c582146 => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:45.747939 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb37a0a-b110-4b8f-a39c-4e8d25d43774 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:45.748471 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb37a0a-b110-4b8f-a39c-4e8d25d43774 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:45.748987 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb37a0a-b110-4b8f-a39c-4e8d25d43774 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:45.752143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 207/434] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:45.753793 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb37a0a-b110-4b8f-a39c-4e8d25d43774 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:45.760021 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:45.773809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589af3a-6507-4d28-b7e9-cc99e3b7e2b4 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:45.834352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863c5bf-506c-4fbd-82f8-9af9d9558838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:45.835384 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863c5bf-506c-4fbd-82f8-9af9d9558838 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:45.835694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863c5bf-506c-4fbd-82f8-9af9d9558838 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:45.836089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863c5bf-506c-4fbd-82f8-9af9d9558838 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:45.836888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863c5bf-506c-4fbd-82f8-9af9d9558838 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:45.837209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863c5bf-506c-4fbd-82f8-9af9d9558838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:45.839057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863c5bf-506c-4fbd-82f8-9af9d9558838 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:45.855798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 228/435] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:40:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:45.855798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137f4f0-acff-4681-9f09-d322c8d471cf None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:45.855798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137f4f0-acff-4681-9f09-d322c8d471cf None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:45.855798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137f4f0-acff-4681-9f09-d322c8d471cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:45.875705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-2137f4f0-acff-4681-9f09-d322c8d471cf None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:45.876930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2137f4f0-acff-4681-9f09-d322c8d471cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:45.975015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb37a0a-b110-4b8f-a39c-4e8d25d43774 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:45.994591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 229/436] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:45.996414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137f4f0-acff-4681-9f09-d322c8d471cf None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:46.044662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcb55b-d882-4fc9-9296-4266b27e3a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:46.045966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcb55b-d882-4fc9-9296-4266b27e3a16 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:46.046681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcb55b-d882-4fc9-9296-4266b27e3a16 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:46.047441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcb55b-d882-4fc9-9296-4266b27e3a16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:46.049006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fcb55b-d882-4fc9-9296-4266b27e3a16 None admin] RBAC: Adding query filter params (group.id=2960f6ca0176451eb54f65211c582146, role.id=6a924f1d619643b3be92bab6fddb3eb2, scope.project.id=21ad3a8335ad492d858142d787614be0) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:46.049740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fcb55b-d882-4fc9-9296-4266b27e3a16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:46.052824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fcb55b-d882-4fc9-9296-4266b27e3a16 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:46.063419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 208/437] 10.222.0.94 () {56 vars in 1376 bytes} [Tue Jul 26 14:40:45 2022] GET /identity/v3/role_assignments?group.id=2960f6ca0176451eb54f65211c582146&scope.project.id=21ad3a8335ad492d858142d787614be0&role.id=6a924f1d619643b3be92bab6fddb3eb2 => generated 611 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:46.064957 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcb55b-d882-4fc9-9296-4266b27e3a16 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:46.150800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:46.169140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137f4f0-acff-4681-9f09-d322c8d471cf None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:46.170044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:46.253962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d2878-3bfe-41ef-bac0-a062f5acb271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:46.255030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d2878-3bfe-41ef-bac0-a062f5acb271 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:46.255551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d2878-3bfe-41ef-bac0-a062f5acb271 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:46.255806 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d2878-3bfe-41ef-bac0-a062f5acb271 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:46.290085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcb55b-d882-4fc9-9296-4266b27e3a16 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:46.372504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca34ec5-19f4-49e5-87d3-4c12016356c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:46.373247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca34ec5-19f4-49e5-87d3-4c12016356c5 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:46.373623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca34ec5-19f4-49e5-87d3-4c12016356c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:46.374054 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca34ec5-19f4-49e5-87d3-4c12016356c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:46.396235 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca34ec5-19f4-49e5-87d3-4c12016356c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-8ca34ec5-19f4-49e5-87d3-4c12016356c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:46.407569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:46.410686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:46.413528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:46.413528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:46.413528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:46.413528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:46.413528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:46.413528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:46.413528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:46.413528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:46.413528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:46.413528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:46.413528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:46.413528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:46.413528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 209/438] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:40:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:46.422671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca34ec5-19f4-49e5-87d3-4c12016356c5 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:46.424651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d2878-3bfe-41ef-bac0-a062f5acb271 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:46.427515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d2878-3bfe-41ef-bac0-a062f5acb271 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:46.582161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:46.616199 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 230/439] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:46.622182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca34ec5-19f4-49e5-87d3-4c12016356c5 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:46.624521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d2878-3bfe-41ef-bac0-a062f5acb271 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:46.642934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:46.684835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022f405-4394-403a-aa6c-2b2b627fa06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:46.685885 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022f405-4394-403a-aa6c-2b2b627fa06e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:46.686020 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022f405-4394-403a-aa6c-2b2b627fa06e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:46.686441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022f405-4394-403a-aa6c-2b2b627fa06e None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:46.687576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a022f405-4394-403a-aa6c-2b2b627fa06e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:46.688095 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a022f405-4394-403a-aa6c-2b2b627fa06e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:46.691233 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a022f405-4394-403a-aa6c-2b2b627fa06e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:46.705673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 210/440] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:40:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:46.708199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10439e29-d4e4-4f7d-aba9-85a0c64382d5 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:46.708655 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10439e29-d4e4-4f7d-aba9-85a0c64382d5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:46.709101 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10439e29-d4e4-4f7d-aba9-85a0c64382d5 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:46.711765 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 211/441] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:46.712951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10439e29-d4e4-4f7d-aba9-85a0c64382d5 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:46.802247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d2878-3bfe-41ef-bac0-a062f5acb271 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:46.855127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6dc33d-818c-494d-9d99-9e9e1444eb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:46.856101 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dc33d-818c-494d-9d99-9e9e1444eb90 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:46.856510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dc33d-818c-494d-9d99-9e9e1444eb90 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:46.856896 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6dc33d-818c-494d-9d99-9e9e1444eb90 None admin] PATH_INFO: `/v3/projects/d0c721e2529541e5966d3c21426e447a` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:46.859180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6dc33d-818c-494d-9d99-9e9e1444eb90 None admin] RBAC: Authorizing `identity:get_project(project_id=d0c721e2529541e5966d3c21426e447a)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:46.861813 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6dc33d-818c-494d-9d99-9e9e1444eb90 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:46.865987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 231/442] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:46 2022] GET /identity/v3/projects/d0c721e2529541e5966d3c21426e447a => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:46.868028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13b9e5-a2fd-4fcf-8902-6fdf6fd819fa None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:46.868532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13b9e5-a2fd-4fcf-8902-6fdf6fd819fa None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:46.869027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13b9e5-a2fd-4fcf-8902-6fdf6fd819fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:46.896017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10439e29-d4e4-4f7d-aba9-85a0c64382d5 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:46.899931 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-ec13b9e5-a2fd-4fcf-8902-6fdf6fd819fa None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:46.901797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ec13b9e5-a2fd-4fcf-8902-6fdf6fd819fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:46.943340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887e7e9-a201-49be-a1a5-27ed1c86a96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:46.944087 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e7e9-a201-49be-a1a5-27ed1c86a96c None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:46.944435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e7e9-a201-49be-a1a5-27ed1c86a96c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:46.944819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887e7e9-a201-49be-a1a5-27ed1c86a96c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:46.989845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:47.002968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:47.025425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 232/443] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:47.026480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13b9e5-a2fd-4fcf-8902-6fdf6fd819fa None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:47.058452 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887e7e9-a201-49be-a1a5-27ed1c86a96c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:47.061199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887e7e9-a201-49be-a1a5-27ed1c86a96c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:47.199105 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13b9e5-a2fd-4fcf-8902-6fdf6fd819fa None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:47.227035 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 212/444] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:40:47.228133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887e7e9-a201-49be-a1a5-27ed1c86a96c None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:47.250461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc3519-2478-4839-bba9-9dde38940960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:47.251512 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc3519-2478-4839-bba9-9dde38940960 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:47.251829 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc3519-2478-4839-bba9-9dde38940960 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:47.252300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc3519-2478-4839-bba9-9dde38940960 None admin] PATH_INFO: `/v3/groups/2960f6ca0176451eb54f65211c582146` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:47.254836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc3519-2478-4839-bba9-9dde38940960 None admin] RBAC: Authorizing `identity:get_group(group_id=2960f6ca0176451eb54f65211c582146)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:47.256680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc3519-2478-4839-bba9-9dde38940960 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:47.259445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 233/445] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:47 2022] GET /identity/v3/groups/2960f6ca0176451eb54f65211c582146 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:47.259445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc3519-2478-4839-bba9-9dde38940960 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:47.356748 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:47.361949 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:47.406050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887e7e9-a201-49be-a1a5-27ed1c86a96c None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:47.430627 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc3519-2478-4839-bba9-9dde38940960 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:47.465815 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e267a1c-0bcf-4d5f-b2ad-547368c24a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:47.466727 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e267a1c-0bcf-4d5f-b2ad-547368c24a4f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:47.467186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e267a1c-0bcf-4d5f-b2ad-547368c24a4f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:47.467683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e267a1c-0bcf-4d5f-b2ad-547368c24a4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:47.480653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812631d-73d2-4de0-98a1-f4b35c9c41ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:47.481442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812631d-73d2-4de0-98a1-f4b35c9c41ed None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:47.481823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812631d-73d2-4de0-98a1-f4b35c9c41ed None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:47.482198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812631d-73d2-4de0-98a1-f4b35c9c41ed None admin] PATH_INFO: `/v3/projects/21ad3a8335ad492d858142d787614be0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:47.484136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812631d-73d2-4de0-98a1-f4b35c9c41ed None admin] RBAC: Authorizing `identity:get_project(project_id=21ad3a8335ad492d858142d787614be0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:47.486044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e267a1c-0bcf-4d5f-b2ad-547368c24a4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:47.486300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812631d-73d2-4de0-98a1-f4b35c9c41ed None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:47.489373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 234/446] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:47 2022] GET /identity/v3/projects/21ad3a8335ad492d858142d787614be0 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:47.490843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812631d-73d2-4de0-98a1-f4b35c9c41ed None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-5e267a1c-0bcf-4d5f-b2ad-547368c24a4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:47.493641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:47.495258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:47.498262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:47.498262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:47.498262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:47.498262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:47.498262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:47.498262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:47.498262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:47.498262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:47.498262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:47.498262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:47.498262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:40:47.498262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:47.498262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 213/447] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:40:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:47.505551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e267a1c-0bcf-4d5f-b2ad-547368c24a4f None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:47.662117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812631d-73d2-4de0-98a1-f4b35c9c41ed None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:47.687023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e267a1c-0bcf-4d5f-b2ad-547368c24a4f None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:47.692733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:47.724819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087393f-029c-43e7-8d0d-188085eac307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:47.725751 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087393f-029c-43e7-8d0d-188085eac307 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:47.726399 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087393f-029c-43e7-8d0d-188085eac307 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:47.727451 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087393f-029c-43e7-8d0d-188085eac307 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:47.729511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087393f-029c-43e7-8d0d-188085eac307 None admin] RBAC: Adding query filter params (group.id=2960f6ca0176451eb54f65211c582146, role.id=eff5cca513be4ed5a9b001c0559fb476, scope.project.id=d0c721e2529541e5966d3c21426e447a) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:47.731581 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087393f-029c-43e7-8d0d-188085eac307 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:47.734411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087393f-029c-43e7-8d0d-188085eac307 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:47.745037 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd60e54-6c7f-4273-a43b-90eb7fecaa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:47.746393 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd60e54-6c7f-4273-a43b-90eb7fecaa3b None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:47.746509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 235/448] 10.222.0.94 () {56 vars in 1376 bytes} [Tue Jul 26 14:40:47 2022] GET /identity/v3/role_assignments?group.id=2960f6ca0176451eb54f65211c582146&scope.project.id=d0c721e2529541e5966d3c21426e447a&role.id=eff5cca513be4ed5a9b001c0559fb476 => generated 611 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:47.746727 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd60e54-6c7f-4273-a43b-90eb7fecaa3b None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:47.747293 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd60e54-6c7f-4273-a43b-90eb7fecaa3b None admin] PATH_INFO: `/v3/groups/2960f6ca0176451eb54f65211c582146` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:47.747679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087393f-029c-43e7-8d0d-188085eac307 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:47.749504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd60e54-6c7f-4273-a43b-90eb7fecaa3b None admin] RBAC: Authorizing `identity:get_group(group_id=2960f6ca0176451eb54f65211c582146)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:47.752315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd60e54-6c7f-4273-a43b-90eb7fecaa3b None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:47.755259 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 214/449] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:47 2022] GET /identity/v3/groups/2960f6ca0176451eb54f65211c582146 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:47.765560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd60e54-6c7f-4273-a43b-90eb7fecaa3b None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:47.769949 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:47.948395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087393f-029c-43e7-8d0d-188085eac307 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:47.966404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd60e54-6c7f-4273-a43b-90eb7fecaa3b None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:48.023812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd486d-4866-48a6-bf84-b7b59ce2a804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:48.025169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd486d-4866-48a6-bf84-b7b59ce2a804 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:48.025662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd486d-4866-48a6-bf84-b7b59ce2a804 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:48.026138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd486d-4866-48a6-bf84-b7b59ce2a804 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:48.027496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd486d-4866-48a6-bf84-b7b59ce2a804 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:48.028761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd486d-4866-48a6-bf84-b7b59ce2a804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:48.030702 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd486d-4866-48a6-bf84-b7b59ce2a804 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:48.047606 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db506c6-e096-4d9f-a78d-836c1720613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:48.048597 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db506c6-e096-4d9f-a78d-836c1720613e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:48.049066 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db506c6-e096-4d9f-a78d-836c1720613e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:48.049551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db506c6-e096-4d9f-a78d-836c1720613e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:48.055858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 236/450] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:40:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:48.056113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db506c6-e096-4d9f-a78d-836c1720613e None admin] RBAC: Adding query filter params (group.id=2960f6ca0176451eb54f65211c582146, role.id=eff5cca513be4ed5a9b001c0559fb476, scope.project.id=21ad3a8335ad492d858142d787614be0) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:48.056113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db506c6-e096-4d9f-a78d-836c1720613e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:48.056113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db506c6-e096-4d9f-a78d-836c1720613e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:48.059970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd486d-4866-48a6-bf84-b7b59ce2a804 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:48.067369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 215/451] 10.222.0.94 () {56 vars in 1376 bytes} [Tue Jul 26 14:40:47 2022] GET /identity/v3/role_assignments?group.id=2960f6ca0176451eb54f65211c582146&scope.project.id=21ad3a8335ad492d858142d787614be0&role.id=eff5cca513be4ed5a9b001c0559fb476 => generated 611 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:40:48.153676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:48.266624 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd486d-4866-48a6-bf84-b7b59ce2a804 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:48.334847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297ea95-353a-410b-9a02-babb41cd6666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:48.335404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297ea95-353a-410b-9a02-babb41cd6666 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:48.335723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297ea95-353a-410b-9a02-babb41cd6666 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:48.336113 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297ea95-353a-410b-9a02-babb41cd6666 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:48.452027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297ea95-353a-410b-9a02-babb41cd6666 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:48.454182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297ea95-353a-410b-9a02-babb41cd6666 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:48.528354 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:48.614814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 237/452] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:40:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 14:40:48.621256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db506c6-e096-4d9f-a78d-836c1720613e None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:48.732797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:48.788940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db506c6-e096-4d9f-a78d-836c1720613e None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:48.837629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf854f80-bddd-4190-92cd-283473d30553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:48.838279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf854f80-bddd-4190-92cd-283473d30553 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:48.838560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf854f80-bddd-4190-92cd-283473d30553 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:48.838836 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf854f80-bddd-4190-92cd-283473d30553 None admin] PATH_INFO: `/v3/projects/382829af20d649b9beb375e3c3cc2d50` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:48.840121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf854f80-bddd-4190-92cd-283473d30553 None admin] RBAC: Authorizing `identity:get_project(project_id=382829af20d649b9beb375e3c3cc2d50)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:48.841662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf854f80-bddd-4190-92cd-283473d30553 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:48.843437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 216/453] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:40:48 2022] GET /identity/v3/projects/382829af20d649b9beb375e3c3cc2d50 => generated 344 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:48.853351 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297ea95-353a-410b-9a02-babb41cd6666 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:49.032785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297ea95-353a-410b-9a02-babb41cd6666 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:49.099126 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bea38-6333-4505-b1d9-78a528355772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:49.100114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bea38-6333-4505-b1d9-78a528355772 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:49.100522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bea38-6333-4505-b1d9-78a528355772 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:49.100927 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bea38-6333-4505-b1d9-78a528355772 None admin] PATH_INFO: `/v3/groups/0215bf55e19443549c0ca04a4a97823a` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:49.102898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1bea38-6333-4505-b1d9-78a528355772 None admin] RBAC: Authorizing `identity:get_group(group_id=0215bf55e19443549c0ca04a4a97823a)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:49.105297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1bea38-6333-4505-b1d9-78a528355772 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:49.108071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 238/454] 10.222.0.94 () {56 vars in 1180 bytes} [Tue Jul 26 14:40:48 2022] GET /identity/v3/groups/0215bf55e19443549c0ca04a4a97823a => generated 229 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:49.116369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf854f80-bddd-4190-92cd-283473d30553 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:49.303553 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf854f80-bddd-4190-92cd-283473d30553 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:49.359441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca883e-8e16-45c5-b943-94ad8cba950a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:49.360459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca883e-8e16-45c5-b943-94ad8cba950a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:49.360969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca883e-8e16-45c5-b943-94ad8cba950a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:49.361307 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca883e-8e16-45c5-b943-94ad8cba950a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:49.362298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca883e-8e16-45c5-b943-94ad8cba950a None admin] RBAC: Adding query filter params (group.id=0215bf55e19443549c0ca04a4a97823a, role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=382829af20d649b9beb375e3c3cc2d50) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:49.363192 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca883e-8e16-45c5-b943-94ad8cba950a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:49.365054 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca883e-8e16-45c5-b943-94ad8cba950a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:49.367686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:49.373902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 217/455] 10.222.0.94 () {56 vars in 1376 bytes} [Tue Jul 26 14:40:49 2022] GET /identity/v3/role_assignments?group.id=0215bf55e19443549c0ca04a4a97823a&scope.project.id=382829af20d649b9beb375e3c3cc2d50&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 611 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:53.091498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9be628-157a-4269-bf4d-63793a05b601 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:53.091498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9be628-157a-4269-bf4d-63793a05b601 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:53.091498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9be628-157a-4269-bf4d-63793a05b601 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:53.094693 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 239/456] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:53.104092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9287e63-b43b-4295-b1b7-1a41192baf9b None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:53.105627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9287e63-b43b-4295-b1b7-1a41192baf9b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:53.106409 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9287e63-b43b-4295-b1b7-1a41192baf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:53.150576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-c9287e63-b43b-4295-b1b7-1a41192baf9b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:53.153371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c9287e63-b43b-4295-b1b7-1a41192baf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:53.332210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 218/457] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:40:53.405150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9be628-157a-4269-bf4d-63793a05b601 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:53.436296 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:53.650328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9be628-157a-4269-bf4d-63793a05b601 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:53.747033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defe004-5790-4a98-8f62-00d1f6d80af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:53.748526 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defe004-5790-4a98-8f62-00d1f6d80af9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:53.749202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defe004-5790-4a98-8f62-00d1f6d80af9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:53.749852 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defe004-5790-4a98-8f62-00d1f6d80af9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:53.777953 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0defe004-5790-4a98-8f62-00d1f6d80af9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:53.780917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0defe004-5790-4a98-8f62-00d1f6d80af9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-0defe004-5790-4a98-8f62-00d1f6d80af9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:40:53.822888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:53.824315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:53.825694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:40:53.825694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:40:53.825694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:40:53.825694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:53.829425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 240/458] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:53.839050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9287e63-b43b-4295-b1b7-1a41192baf9b None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:54.072436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9287e63-b43b-4295-b1b7-1a41192baf9b None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:54.136347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:54.150471 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52856081-63c2-4f16-bbc6-94a69cea3a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:54.151142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52856081-63c2-4f16-bbc6-94a69cea3a1b None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:54.151519 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52856081-63c2-4f16-bbc6-94a69cea3a1b None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:54.155645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52856081-63c2-4f16-bbc6-94a69cea3a1b None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:54.155645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52856081-63c2-4f16-bbc6-94a69cea3a1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:54.155645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52856081-63c2-4f16-bbc6-94a69cea3a1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:54.155645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52856081-63c2-4f16-bbc6-94a69cea3a1b None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:54.176990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 219/459] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:54.185953 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defe004-5790-4a98-8f62-00d1f6d80af9 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:54.254538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99e908-fd7e-46d1-bec6-7c434b4288c7 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:54.255125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99e908-fd7e-46d1-bec6-7c434b4288c7 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:54.255655 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99e908-fd7e-46d1-bec6-7c434b4288c7 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:54.259374 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 220/460] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:54.270264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff47b0-1783-4a8e-b695-7ba228a7125a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:54.271067 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff47b0-1783-4a8e-b695-7ba228a7125a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:54.271792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff47b0-1783-4a8e-b695-7ba228a7125a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:54.324328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-e7ff47b0-1783-4a8e-b695-7ba228a7125a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:54.332209 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e7ff47b0-1783-4a8e-b695-7ba228a7125a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:54.400029 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:54.498518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defe004-5790-4a98-8f62-00d1f6d80af9 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:54.575429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53b291-c2b3-4cbb-9e2c-ad2f94625e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:54.577334 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53b291-c2b3-4cbb-9e2c-ad2f94625e20 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:54.578129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53b291-c2b3-4cbb-9e2c-ad2f94625e20 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:54.578823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53b291-c2b3-4cbb-9e2c-ad2f94625e20 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:54.580724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53b291-c2b3-4cbb-9e2c-ad2f94625e20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:54.585624 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53b291-c2b3-4cbb-9e2c-ad2f94625e20 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:54.605563 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 221/461] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:54.608577 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91ea10-4143-4469-856c-1868d8188045 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:54.609542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91ea10-4143-4469-856c-1868d8188045 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:54.610156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91ea10-4143-4469-856c-1868d8188045 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:54.614422 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 222/462] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:54.618067 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c593f-6d4d-43ef-ac9a-db6c905812bc None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:54.618959 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c593f-6d4d-43ef-ac9a-db6c905812bc None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:54.619884 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c593f-6d4d-43ef-ac9a-db6c905812bc None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:54.623228 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 223/463] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:54.630358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534031-69a0-4797-b1c4-2697d08ea056 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:54.630358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534031-69a0-4797-b1c4-2697d08ea056 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:54.630358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534031-69a0-4797-b1c4-2697d08ea056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:54.663192 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-68534031-69a0-4797-b1c4-2697d08ea056 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:54.665440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-68534031-69a0-4797-b1c4-2697d08ea056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:54.793992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 224/464] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:54.795715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf74af-c28d-4902-b66b-ae408ad5bc60 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:54.796164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf74af-c28d-4902-b66b-ae408ad5bc60 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:54.796610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf74af-c28d-4902-b66b-ae408ad5bc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:54.819812 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-43cf74af-c28d-4902-b66b-ae408ad5bc60 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:54.821091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-43cf74af-c28d-4902-b66b-ae408ad5bc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:54.886269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 241/465] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:40:54 2022] POST /identity/v3/users => generated 246 bytes in 701 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:40:54.887489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53b291-c2b3-4cbb-9e2c-ad2f94625e20 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:54.894418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:54.965976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 225/466] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:54.966472 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf74af-c28d-4902-b66b-ae408ad5bc60 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:55.078830 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53b291-c2b3-4cbb-9e2c-ad2f94625e20 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:55.116540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:55.140550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc4814-b852-48cc-b554-c8feea1e911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:55.141210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4814-b852-48cc-b554-c8feea1e911d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:55.141485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4814-b852-48cc-b554-c8feea1e911d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:55.141930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc4814-b852-48cc-b554-c8feea1e911d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:55.165303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc4814-b852-48cc-b554-c8feea1e911d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:55.168182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc4814-b852-48cc-b554-c8feea1e911d None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-9fbc4814-b852-48cc-b554-c8feea1e911d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:40:55.206925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:55.209651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:55.211355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:40:55.211355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:40:55.211355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:40:55.211355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:55.211355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 242/467] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:55.211355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc4814-b852-48cc-b554-c8feea1e911d None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:55.228244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf74af-c28d-4902-b66b-ae408ad5bc60 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:55.301295 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d9f54-c46d-4669-80e7-de0f0c129a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:55.302868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d9f54-c46d-4669-80e7-de0f0c129a1e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:55.303615 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d9f54-c46d-4669-80e7-de0f0c129a1e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:55.303998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d9f54-c46d-4669-80e7-de0f0c129a1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:55.323086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d9f54-c46d-4669-80e7-de0f0c129a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:55.325462 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d9f54-c46d-4669-80e7-de0f0c129a1e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-091d9f54-c46d-4669-80e7-de0f0c129a1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:40:55.351612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:55.352720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:55.354127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:40:55.354127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:40:55.354127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:40:55.354127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:55.355894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 226/468] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:55.358453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed63a0-5224-42de-a478-8b6f3de857c9 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:55.360437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed63a0-5224-42de-a478-8b6f3de857c9 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:55.360805 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed63a0-5224-42de-a478-8b6f3de857c9 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:55.364794 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 227/469] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:55.365683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed63a0-5224-42de-a478-8b6f3de857c9 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:55.398597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc4814-b852-48cc-b554-c8feea1e911d None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:55.436573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:55.470977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d67bc0-7c3c-458c-a1bc-924ffd5ca00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:55.472169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d67bc0-7c3c-458c-a1bc-924ffd5ca00a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:55.472674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d67bc0-7c3c-458c-a1bc-924ffd5ca00a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:55.473163 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d67bc0-7c3c-458c-a1bc-924ffd5ca00a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:55.488650 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d67bc0-7c3c-458c-a1bc-924ffd5ca00a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:55.490221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d67bc0-7c3c-458c-a1bc-924ffd5ca00a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-a4d67bc0-7c3c-458c-a1bc-924ffd5ca00a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:40:55.511279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:55.512244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:55.513247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:40:55.513247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:40:55.513247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:40:55.513247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:55.513247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 243/470] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:55.513597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d67bc0-7c3c-458c-a1bc-924ffd5ca00a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:55.559007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:55.570549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed63a0-5224-42de-a478-8b6f3de857c9 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:55.633653 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff504fe-0739-42de-9975-aa46a5ef00b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:55.634399 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff504fe-0739-42de-9975-aa46a5ef00b0 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:55.634687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff504fe-0739-42de-9975-aa46a5ef00b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:55.635027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff504fe-0739-42de-9975-aa46a5ef00b0 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:55.636247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff504fe-0739-42de-9975-aa46a5ef00b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:55.636576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff504fe-0739-42de-9975-aa46a5ef00b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:55.638440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff504fe-0739-42de-9975-aa46a5ef00b0 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:55.658089 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 228/471] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:55.660104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55ed7-fda9-4f34-87f0-b4757e577e72 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:55.660399 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55ed7-fda9-4f34-87f0-b4757e577e72 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:55.660679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55ed7-fda9-4f34-87f0-b4757e577e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:55.693658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-73f55ed7-fda9-4f34-87f0-b4757e577e72 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:55.700748 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-73f55ed7-fda9-4f34-87f0-b4757e577e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:55.712704 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d67bc0-7c3c-458c-a1bc-924ffd5ca00a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:55.770476 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944ae42-b9d8-4f35-8ccc-af5456d4be2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:55.771197 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944ae42-b9d8-4f35-8ccc-af5456d4be2a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:55.771492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944ae42-b9d8-4f35-8ccc-af5456d4be2a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:55.772019 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944ae42-b9d8-4f35-8ccc-af5456d4be2a None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:55.772901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944ae42-b9d8-4f35-8ccc-af5456d4be2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:55.773240 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944ae42-b9d8-4f35-8ccc-af5456d4be2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:55.775115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944ae42-b9d8-4f35-8ccc-af5456d4be2a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:55.793957 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 244/472] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:55.793957 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944ae42-b9d8-4f35-8ccc-af5456d4be2a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:55.817236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 229/473] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:40:55.819061 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f55ed7-fda9-4f34-87f0-b4757e577e72 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:55.874591 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:55.957115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:55.986041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944ae42-b9d8-4f35-8ccc-af5456d4be2a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:56.004838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f55ed7-fda9-4f34-87f0-b4757e577e72 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:56.050401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c7af1-72aa-44f5-bdc2-97535523d7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:56.051213 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c7af1-72aa-44f5-bdc2-97535523d7b4 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:56.051592 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c7af1-72aa-44f5-bdc2-97535523d7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:56.051880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c7af1-72aa-44f5-bdc2-97535523d7b4 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:56.052653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c7af1-72aa-44f5-bdc2-97535523d7b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:56.053003 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c7af1-72aa-44f5-bdc2-97535523d7b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:56.054515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c7af1-72aa-44f5-bdc2-97535523d7b4 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:56.062001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550c5cf-609b-4e03-9c2b-beacfd9b2a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:56.064804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550c5cf-609b-4e03-9c2b-beacfd9b2a63 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:56.066372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550c5cf-609b-4e03-9c2b-beacfd9b2a63 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:56.066972 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550c5cf-609b-4e03-9c2b-beacfd9b2a63 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:56.069024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550c5cf-609b-4e03-9c2b-beacfd9b2a63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:56.070789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550c5cf-609b-4e03-9c2b-beacfd9b2a63 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:56.078261 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 245/474] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:56.078261 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c7af1-72aa-44f5-bdc2-97535523d7b4 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:56.179660 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 230/475] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:40:55 2022] POST /identity/v3/users => generated 248 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:40:56.193116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550c5cf-609b-4e03-9c2b-beacfd9b2a63 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:56.280210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c7af1-72aa-44f5-bdc2-97535523d7b4 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:56.369023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc5d67-69d5-42b4-bdc9-5c076de87df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:56.370400 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc5d67-69d5-42b4-bdc9-5c076de87df9 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:56.371039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc5d67-69d5-42b4-bdc9-5c076de87df9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:56.371705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc5d67-69d5-42b4-bdc9-5c076de87df9 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:56.373278 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc5d67-69d5-42b4-bdc9-5c076de87df9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:56.373907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:56.377487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc5d67-69d5-42b4-bdc9-5c076de87df9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:56.416695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550c5cf-609b-4e03-9c2b-beacfd9b2a63 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:56.480976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2baf4-9421-4abe-a546-5ce7fc12dfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:56.482053 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2baf4-9421-4abe-a546-5ce7fc12dfc9 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:56.482604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2baf4-9421-4abe-a546-5ce7fc12dfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:56.483118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2baf4-9421-4abe-a546-5ce7fc12dfc9 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:56.484701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2baf4-9421-4abe-a546-5ce7fc12dfc9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:56.488754 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2baf4-9421-4abe-a546-5ce7fc12dfc9 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:56.590647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.sql.core [None req-06c2baf4-9421-4abe-a546-5ce7fc12dfc9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 14:40:56.590647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 14:40:56.590647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [parameters: {'id': 'ee86f5f841dc4dce820ba84f5cd81973', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 14:40:56.590647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92209) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 14:40:56.595627 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:56.611843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 246/476] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:40:56 2022] POST /identity/v3/users => generated 248 bytes in 530 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:40:56.614017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc5d67-69d5-42b4-bdc9-5c076de87df9 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-06c2baf4-9421-4abe-a546-5ce7fc12dfc9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application conn.query(q) Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application result.read() Jul 26 14:40:56.626934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return ref.to_dict() Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 14:40:56.628871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application next(self.gen) Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application yield resource Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application next(self.gen) Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise self.value Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application session.commit() Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 14:40:56.630684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self.session.flush() Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application compat.raise_( Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 14:40:56.632505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 14:40:56.634223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application conn.query(q) Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application result.read() Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 14:40:56.637570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [parameters: {'id': 'ee86f5f841dc4dce820ba84f5cd81973', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:56.651189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 14:40:56.666880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:56.669520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 231/477] 10.222.0.94 () {60 vars in 1131 bytes} [Tue Jul 26 14:40:56 2022] POST /identity/v3/roles => generated 144 bytes in 448 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:56.669520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2baf4-9421-4abe-a546-5ce7fc12dfc9 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:56.930493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2baf4-9421-4abe-a546-5ce7fc12dfc9 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:56.941782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc5d67-69d5-42b4-bdc9-5c076de87df9 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:57.001413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c64ae-3fa6-4efc-a73e-c1c0e15b5d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:57.002469 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c64ae-3fa6-4efc-a73e-c1c0e15b5d25 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:57.003498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c64ae-3fa6-4efc-a73e-c1c0e15b5d25 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:57.004046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c64ae-3fa6-4efc-a73e-c1c0e15b5d25 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:57.005401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c64ae-3fa6-4efc-a73e-c1c0e15b5d25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:57.007908 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c64ae-3fa6-4efc-a73e-c1c0e15b5d25 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:57.011933 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bca735-3e42-469e-99ac-cc66b0778b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:57.013392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bca735-3e42-469e-99ac-cc66b0778b74 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:57.013993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bca735-3e42-469e-99ac-cc66b0778b74 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:57.014610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bca735-3e42-469e-99ac-cc66b0778b74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:57.036930 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:57.043803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bca735-3e42-469e-99ac-cc66b0778b74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-c5bca735-3e42-469e-99ac-cc66b0778b74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:57.054820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:57.056737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:57.058403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:57.058403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:57.058403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:57.058403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:57.058403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:57.058403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:57.058403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:57.058403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:57.058403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:57.058403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:57.058403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 14:40:57.058403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:40:57.062366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 232/478] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:40:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:57.070472 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bca735-3e42-469e-99ac-cc66b0778b74 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:57.113838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 247/479] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:40:56 2022] POST /identity/v3/users => generated 249 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:40:57.353156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bca735-3e42-469e-99ac-cc66b0778b74 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:57.429470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a13f1-07ca-452a-af16-5255cba7a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:57.430491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a13f1-07ca-452a-af16-5255cba7a210 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:57.430491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a13f1-07ca-452a-af16-5255cba7a210 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:57.430778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a13f1-07ca-452a-af16-5255cba7a210 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:57.431636 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a13f1-07ca-452a-af16-5255cba7a210 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:57.431953 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a13f1-07ca-452a-af16-5255cba7a210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:57.433405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a13f1-07ca-452a-af16-5255cba7a210 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:57.447122 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 233/480] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:40:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:58.052685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a2e56-f0e2-4b7d-a342-a2674b3ccb39 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:58.053757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a2e56-f0e2-4b7d-a342-a2674b3ccb39 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:58.053757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a2e56-f0e2-4b7d-a342-a2674b3ccb39 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:58.056653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 248/481] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:40:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:40:58.064709 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73fec5-9411-49f3-b95d-bfeaed3b4e1e None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:58.065721 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73fec5-9411-49f3-b95d-bfeaed3b4e1e None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:58.066006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73fec5-9411-49f3-b95d-bfeaed3b4e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:58.102136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-4c73fec5-9411-49f3-b95d-bfeaed3b4e1e None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:40:58.103370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4c73fec5-9411-49f3-b95d-bfeaed3b4e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:40:58.249977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:58.271639 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 234/482] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:40:58.335390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a2e56-f0e2-4b7d-a342-a2674b3ccb39 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:58.439636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:58.566787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a2e56-f0e2-4b7d-a342-a2674b3ccb39 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:58.657051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b20ca-37c5-4eea-97bb-c1a846bfd5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:58.658079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b20ca-37c5-4eea-97bb-c1a846bfd5b0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:58.658566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b20ca-37c5-4eea-97bb-c1a846bfd5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:58.659745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b20ca-37c5-4eea-97bb-c1a846bfd5b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:58.689080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b20ca-37c5-4eea-97bb-c1a846bfd5b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-764b20ca-37c5-4eea-97bb-c1a846bfd5b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:40:58.700685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:58.702491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:58.704104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:58.704104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:58.704104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:58.704104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:58.704104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:40:58.704104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:40:58.704104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:40:58.704104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:40:58.704104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:40:58.704104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:40:58.704104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:40:58.704104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:58.704104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 249/483] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:40:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:58.713428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73fec5-9411-49f3-b95d-bfeaed3b4e1e None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:58.991494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73fec5-9411-49f3-b95d-bfeaed3b4e1e None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:59.064758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30f3c1-966f-4d6a-b64e-e19076ba84bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:59.066367 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30f3c1-966f-4d6a-b64e-e19076ba84bb None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:59.066814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30f3c1-966f-4d6a-b64e-e19076ba84bb None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:59.067283 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30f3c1-966f-4d6a-b64e-e19076ba84bb None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:59.068643 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30f3c1-966f-4d6a-b64e-e19076ba84bb None admin] RBAC: Adding query filter params (name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:59.069163 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30f3c1-966f-4d6a-b64e-e19076ba84bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:59.071804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30f3c1-966f-4d6a-b64e-e19076ba84bb None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:59.092998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 235/484] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:40:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:59.105361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b20ca-37c5-4eea-97bb-c1a846bfd5b0 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:59.344918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b20ca-37c5-4eea-97bb-c1a846bfd5b0 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:59.368222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:59.407266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068423e-43ab-4d10-af0d-50cea1cd9589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:59.408378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068423e-43ab-4d10-af0d-50cea1cd9589 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:59.409640 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068423e-43ab-4d10-af0d-50cea1cd9589 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:59.410148 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068423e-43ab-4d10-af0d-50cea1cd9589 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:59.430355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068423e-43ab-4d10-af0d-50cea1cd9589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:59.436300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068423e-43ab-4d10-af0d-50cea1cd9589 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-8068423e-43ab-4d10-af0d-50cea1cd9589 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:40:59.464758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:40:59.466408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:40:59.469324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:40:59.469324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:40:59.469324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:40:59.469324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:40:59.469324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 250/485] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:40:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:40:59.477369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30f3c1-966f-4d6a-b64e-e19076ba84bb None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:40:59.652685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:40:59.773510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30f3c1-966f-4d6a-b64e-e19076ba84bb None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:40:59.851267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e371063-441e-4816-ac14-78d4aca4e88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:40:59.852431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e371063-441e-4816-ac14-78d4aca4e88e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:40:59.852950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e371063-441e-4816-ac14-78d4aca4e88e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:40:59.853457 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e371063-441e-4816-ac14-78d4aca4e88e None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:40:59.854885 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e371063-441e-4816-ac14-78d4aca4e88e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:40:59.855451 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e371063-441e-4816-ac14-78d4aca4e88e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:40:59.858214 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e371063-441e-4816-ac14-78d4aca4e88e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:40:59.885017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 236/486] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:40:59.891369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068423e-43ab-4d10-af0d-50cea1cd9589 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:00.150407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068423e-43ab-4d10-af0d-50cea1cd9589 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:00.237531 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef221b10-7719-4cc0-97ac-3de4db93ffad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:00.238654 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef221b10-7719-4cc0-97ac-3de4db93ffad None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:00.239163 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef221b10-7719-4cc0-97ac-3de4db93ffad None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:00.239721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef221b10-7719-4cc0-97ac-3de4db93ffad None admin] PATH_INFO: `/v3/users/nova` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:00.260359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef221b10-7719-4cc0-97ac-3de4db93ffad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:00.263244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef221b10-7719-4cc0-97ac-3de4db93ffad None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-ef221b10-7719-4cc0-97ac-3de4db93ffad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:00.294857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:00.296662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:00.298895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:00.298895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:00.298895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:00.298895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:00.298895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:00.298895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:00.298895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:00.298895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:00.298895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:41:00.298895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:00.298895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 251/487] 10.222.0.94 () {56 vars in 1093 bytes} [Tue Jul 26 14:40:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:00.309282 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e371063-441e-4816-ac14-78d4aca4e88e None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:00.325291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76af5d8-a24f-4c50-b48e-da22dc7ebdbb None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:00.325784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76af5d8-a24f-4c50-b48e-da22dc7ebdbb None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:00.326310 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76af5d8-a24f-4c50-b48e-da22dc7ebdbb None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:00.329401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 252/488] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:41:00.339771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895b2cf-a7f0-4e54-a703-98f6affbf86b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:00.340210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895b2cf-a7f0-4e54-a703-98f6affbf86b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:00.340663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895b2cf-a7f0-4e54-a703-98f6affbf86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:00.398160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-f895b2cf-a7f0-4e54-a703-98f6affbf86b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:00.400310 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f895b2cf-a7f0-4e54-a703-98f6affbf86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:00.456803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:00.554894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 253/489] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:41:00.613337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e371063-441e-4816-ac14-78d4aca4e88e None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:00.617656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895b2cf-a7f0-4e54-a703-98f6affbf86b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:00.632998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:00.659566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49025893-f25d-4086-8a51-e961b2a84370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:00.660672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49025893-f25d-4086-8a51-e961b2a84370 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:00.661361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49025893-f25d-4086-8a51-e961b2a84370 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:00.661743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49025893-f25d-4086-8a51-e961b2a84370 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:00.662953 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49025893-f25d-4086-8a51-e961b2a84370 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:00.663389 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49025893-f25d-4086-8a51-e961b2a84370 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:00.665352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49025893-f25d-4086-8a51-e961b2a84370 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:00.722025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 237/490] 10.222.0.94 () {56 vars in 1133 bytes} [Tue Jul 26 14:41:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:00.723530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49025893-f25d-4086-8a51-e961b2a84370 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:00.856064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49025893-f25d-4086-8a51-e961b2a84370 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:00.903227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1bc5f-e828-4d91-b905-6a111a799d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:00.903912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1bc5f-e828-4d91-b905-6a111a799d60 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:00.904205 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1bc5f-e828-4d91-b905-6a111a799d60 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:00.904490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1bc5f-e828-4d91-b905-6a111a799d60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:00.964359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:01.006875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1bc5f-e828-4d91-b905-6a111a799d60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:01.008494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895b2cf-a7f0-4e54-a703-98f6affbf86b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:01.008864 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1bc5f-e828-4d91-b905-6a111a799d60 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:01.088962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff7b4f-327e-4040-bea7-1ec812717e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:01.090061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff7b4f-327e-4040-bea7-1ec812717e99 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:01.090549 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff7b4f-327e-4040-bea7-1ec812717e99 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:01.091045 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff7b4f-327e-4040-bea7-1ec812717e99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:01.105316 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff7b4f-327e-4040-bea7-1ec812717e99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-5eff7b4f-327e-4040-bea7-1ec812717e99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:01.112487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:01.113718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:01.114495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:01.114495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:01.114495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:01.114495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:01.114495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:01.114495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:01.114495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:01.114495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:01.114495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:01.114495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:01.114495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:01.114495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:01.119044 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 254/491] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 502 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:41:01.129823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff7b4f-327e-4040-bea7-1ec812717e99 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:01.182956 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 238/492] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 14:41:01.191931 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:01.193180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1bc5f-e828-4d91-b905-6a111a799d60 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:01.303840 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:01.308681 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff7b4f-327e-4040-bea7-1ec812717e99 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:01.400968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1bc5f-e828-4d91-b905-6a111a799d60 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:01.419421 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13c743-fd39-4e8c-bb04-99e71776ac3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:01.420127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13c743-fd39-4e8c-bb04-99e71776ac3e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:01.420404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13c743-fd39-4e8c-bb04-99e71776ac3e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:01.420725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13c743-fd39-4e8c-bb04-99e71776ac3e None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:01.421620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13c743-fd39-4e8c-bb04-99e71776ac3e None admin] RBAC: Adding query filter params (name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:01.421923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13c743-fd39-4e8c-bb04-99e71776ac3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:01.423407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13c743-fd39-4e8c-bb04-99e71776ac3e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:01.434083 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 255/493] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:01.435433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635eb7a-100e-4947-a2e1-2130bdeaf017 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:01.435745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635eb7a-100e-4947-a2e1-2130bdeaf017 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:01.436024 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635eb7a-100e-4947-a2e1-2130bdeaf017 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:01.439529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 256/494] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:01.439529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e9730-90f8-468e-b85d-8a5826991e09 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:01.439529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e9730-90f8-468e-b85d-8a5826991e09 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:01.439529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e9730-90f8-468e-b85d-8a5826991e09 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:01.445778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 257/495] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:01.445778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902c5c2-6b28-4a02-bb66-e6b8cd196d8d None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:01.445778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902c5c2-6b28-4a02-bb66-e6b8cd196d8d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:01.445778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902c5c2-6b28-4a02-bb66-e6b8cd196d8d None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:01.445778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 258/496] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:01.445778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902c5c2-6b28-4a02-bb66-e6b8cd196d8d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:01.457637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda84dc-ef0a-453c-b0e7-7f35bb29ba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:01.458632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda84dc-ef0a-453c-b0e7-7f35bb29ba35 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:01.459154 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda84dc-ef0a-453c-b0e7-7f35bb29ba35 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:01.459617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda84dc-ef0a-453c-b0e7-7f35bb29ba35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:01.474538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda84dc-ef0a-453c-b0e7-7f35bb29ba35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:01.476389 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda84dc-ef0a-453c-b0e7-7f35bb29ba35 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-4cda84dc-ef0a-453c-b0e7-7f35bb29ba35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:01.501648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:01.504029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:01.506984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:01.506984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:01.506984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:01.506984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:01.506984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 239/497] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:01.506984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c892585-6edb-4df9-89da-cb1595f05188 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:01.506984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c892585-6edb-4df9-89da-cb1595f05188 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:01.506984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c892585-6edb-4df9-89da-cb1595f05188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:01.534890 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-9c892585-6edb-4df9-89da-cb1595f05188 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:01.537280 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9c892585-6edb-4df9-89da-cb1595f05188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:01.582277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:01.624007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902c5c2-6b28-4a02-bb66-e6b8cd196d8d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:01.647252 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 240/498] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:01.648664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be4496-a6e6-45d3-bd37-d7ab93f8a933 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:01.648934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be4496-a6e6-45d3-bd37-d7ab93f8a933 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:01.649225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be4496-a6e6-45d3-bd37-d7ab93f8a933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:01.670968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-23be4496-a6e6-45d3-bd37-d7ab93f8a933 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:01.672420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-23be4496-a6e6-45d3-bd37-d7ab93f8a933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:01.677756 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c4461-b88a-4a1a-88ce-176cd1e2d6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:01.678800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c4461-b88a-4a1a-88ce-176cd1e2d6a6 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:01.679255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c4461-b88a-4a1a-88ce-176cd1e2d6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:01.679723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c4461-b88a-4a1a-88ce-176cd1e2d6a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:01.699450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c4461-b88a-4a1a-88ce-176cd1e2d6a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:01.701708 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c4461-b88a-4a1a-88ce-176cd1e2d6a6 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-022c4461-b88a-4a1a-88ce-176cd1e2d6a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:01.719349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:01.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:01.722558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:01.722558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:01.722558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:01.722558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:01.722558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 259/499] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:01.722558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f35b3d-8236-44cc-8a62-4aa1b60e839b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:01.722558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f35b3d-8236-44cc-8a62-4aa1b60e839b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:01.722949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f35b3d-8236-44cc-8a62-4aa1b60e839b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:01.745235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c6f35b3d-8236-44cc-8a62-4aa1b60e839b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:01.746605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c6f35b3d-8236-44cc-8a62-4aa1b60e839b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:01.757865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:01.780847 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 241/500] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:01.781726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be4496-a6e6-45d3-bd37-d7ab93f8a933 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:01.837352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 260/501] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:01.837957 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f35b3d-8236-44cc-8a62-4aa1b60e839b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:01.917799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be4496-a6e6-45d3-bd37-d7ab93f8a933 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:01.952394 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:01.968700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3ff29-4616-4440-a269-595acbfef338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:01.969511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3ff29-4616-4440-a269-595acbfef338 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:01.969709 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3ff29-4616-4440-a269-595acbfef338 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:01.969945 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3ff29-4616-4440-a269-595acbfef338 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:01.970751 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3ff29-4616-4440-a269-595acbfef338 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:01.971155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3ff29-4616-4440-a269-595acbfef338 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:01.972928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3ff29-4616-4440-a269-595acbfef338 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:01.989814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 242/502] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:01.991356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3ff29-4616-4440-a269-595acbfef338 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:02.041210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f35b3d-8236-44cc-8a62-4aa1b60e839b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:02.061521 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:02.102535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a8749-a2cf-48b3-9301-05bc728d4aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:02.103373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a8749-a2cf-48b3-9301-05bc728d4aa4 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:02.103927 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a8749-a2cf-48b3-9301-05bc728d4aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:02.104281 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a8749-a2cf-48b3-9301-05bc728d4aa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:02.117753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a8749-a2cf-48b3-9301-05bc728d4aa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:02.119287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a8749-a2cf-48b3-9301-05bc728d4aa4 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-7c3a8749-a2cf-48b3-9301-05bc728d4aa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:02.143666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:02.145485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:02.147423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:02.147423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:02.147423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:02.147423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:02.147423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 261/503] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:02.148012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a8749-a2cf-48b3-9301-05bc728d4aa4 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:02.159857 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3ff29-4616-4440-a269-595acbfef338 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:02.205608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9218c2-369c-4635-913a-5c0db333bf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:02.206358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9218c2-369c-4635-913a-5c0db333bf0e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:02.206597 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9218c2-369c-4635-913a-5c0db333bf0e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:02.206860 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9218c2-369c-4635-913a-5c0db333bf0e None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:02.207659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9218c2-369c-4635-913a-5c0db333bf0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:02.207974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9218c2-369c-4635-913a-5c0db333bf0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:02.209402 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9218c2-369c-4635-913a-5c0db333bf0e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:02.230739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 243/504] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:02.231841 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9218c2-369c-4635-913a-5c0db333bf0e None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:02.320352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a8749-a2cf-48b3-9301-05bc728d4aa4 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:02.340540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:02.368471 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4c72c-ff61-4602-871f-93e9d81c3811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:02.369055 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4c72c-ff61-4602-871f-93e9d81c3811 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:02.369349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4c72c-ff61-4602-871f-93e9d81c3811 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:02.369629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4c72c-ff61-4602-871f-93e9d81c3811 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:02.383958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4c72c-ff61-4602-871f-93e9d81c3811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-e8c4c72c-ff61-4602-871f-93e9d81c3811 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:02.389158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:02.391010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:02.392831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:02.392831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:02.392831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:02.392831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:02.392831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:02.392831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:02.392831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:02.392831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:02.392831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:02.392831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:02.392831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:02.392831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:02.398395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 262/505] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:02.399684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4c72c-ff61-4602-871f-93e9d81c3811 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:02.409444 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9218c2-369c-4635-913a-5c0db333bf0e None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:02.412779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:02.465394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead1b59-9f0e-4a01-a17a-b2937ad6db59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:02.466513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead1b59-9f0e-4a01-a17a-b2937ad6db59 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:02.466989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead1b59-9f0e-4a01-a17a-b2937ad6db59 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:02.467502 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead1b59-9f0e-4a01-a17a-b2937ad6db59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:02.489161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead1b59-9f0e-4a01-a17a-b2937ad6db59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-fead1b59-9f0e-4a01-a17a-b2937ad6db59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:02.498678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:02.500718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:02.502368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:02.502368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:02.502368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:02.502368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:02.502368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:02.502368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:02.502368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:02.502368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:02.502368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:02.502368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:02.502368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:02.502368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:02.502368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 244/506] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:02.502368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead1b59-9f0e-4a01-a17a-b2937ad6db59 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:02.561127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4c72c-ff61-4602-871f-93e9d81c3811 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:02.620348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f204f4c-7cdb-4684-bdcc-9b16c96aba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:02.621364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f204f4c-7cdb-4684-bdcc-9b16c96aba19 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:02.621828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f204f4c-7cdb-4684-bdcc-9b16c96aba19 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:02.622311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f204f4c-7cdb-4684-bdcc-9b16c96aba19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:02.636471 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f204f4c-7cdb-4684-bdcc-9b16c96aba19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:02.638290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f204f4c-7cdb-4684-bdcc-9b16c96aba19 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-7f204f4c-7cdb-4684-bdcc-9b16c96aba19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:02.663419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:02.664653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:02.665497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:02.665497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:02.665497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:02.665497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:02.665497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:02.665497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:02.665497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:02.665497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:02.666150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 263/507] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:02.667141 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f204f4c-7cdb-4684-bdcc-9b16c96aba19 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:02.697219 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead1b59-9f0e-4a01-a17a-b2937ad6db59 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:02.743844 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:02.752480 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045f8ae-d935-4348-98de-56d182815519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:02.753589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045f8ae-d935-4348-98de-56d182815519 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:02.753903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045f8ae-d935-4348-98de-56d182815519 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:02.759860 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045f8ae-d935-4348-98de-56d182815519 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:02.759860 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d045f8ae-d935-4348-98de-56d182815519 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:02.759860 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d045f8ae-d935-4348-98de-56d182815519 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:02.759860 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d045f8ae-d935-4348-98de-56d182815519 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:02.775310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 245/508] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:02.776498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045f8ae-d935-4348-98de-56d182815519 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:02.813215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:02.855049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f204f4c-7cdb-4684-bdcc-9b16c96aba19 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:02.903678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01168a0a-b993-43c8-9f11-89669ffb1d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:02.904301 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01168a0a-b993-43c8-9f11-89669ffb1d61 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:02.904587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01168a0a-b993-43c8-9f11-89669ffb1d61 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:02.904893 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01168a0a-b993-43c8-9f11-89669ffb1d61 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:02.917229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01168a0a-b993-43c8-9f11-89669ffb1d61 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:02.918712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01168a0a-b993-43c8-9f11-89669ffb1d61 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:02.930477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045f8ae-d935-4348-98de-56d182815519 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-01168a0a-b993-43c8-9f11-89669ffb1d61 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:02.932468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:02.933969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:02.936803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:02.936803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:02.936803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:02.936803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:02.936803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:02.936803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:02.936803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:02.936803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:02.936803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:41:02.936803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:02.936803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 264/509] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:41:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:02.936803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01168a0a-b993-43c8-9f11-89669ffb1d61 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:02.979429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80040a8b-89b2-4007-88c4-934ba0b19863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:02.980146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80040a8b-89b2-4007-88c4-934ba0b19863 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:02.980343 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80040a8b-89b2-4007-88c4-934ba0b19863 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:02.980629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80040a8b-89b2-4007-88c4-934ba0b19863 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:02.981435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80040a8b-89b2-4007-88c4-934ba0b19863 None admin] RBAC: Adding query filter params (name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:02.981742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80040a8b-89b2-4007-88c4-934ba0b19863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:02.983235 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80040a8b-89b2-4007-88c4-934ba0b19863 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:02.995642 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 246/510] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:02.996510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80040a8b-89b2-4007-88c4-934ba0b19863 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:03.073828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:03.082789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01168a0a-b993-43c8-9f11-89669ffb1d61 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:03.124393 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7971150-4321-43de-9827-04d6c115ee7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:03.125223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7971150-4321-43de-9827-04d6c115ee7b None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:03.125593 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7971150-4321-43de-9827-04d6c115ee7b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:03.126003 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7971150-4321-43de-9827-04d6c115ee7b None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:03.126994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7971150-4321-43de-9827-04d6c115ee7b None admin] RBAC: Adding query filter params (name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:03.127537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7971150-4321-43de-9827-04d6c115ee7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:03.129439 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7971150-4321-43de-9827-04d6c115ee7b None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:03.139615 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:03.140347 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 265/511] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:03.141191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7971150-4321-43de-9827-04d6c115ee7b None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:03.170094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80040a8b-89b2-4007-88c4-934ba0b19863 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:03.226863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c5689-ed49-4e31-9d21-462c221cd102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:03.227677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c5689-ed49-4e31-9d21-462c221cd102 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:03.228072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c5689-ed49-4e31-9d21-462c221cd102 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:03.228482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c5689-ed49-4e31-9d21-462c221cd102 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:03.230061 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c5689-ed49-4e31-9d21-462c221cd102 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:03.230446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c5689-ed49-4e31-9d21-462c221cd102 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:03.233190 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c5689-ed49-4e31-9d21-462c221cd102 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:03.259435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 247/512] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:03.260857 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c5689-ed49-4e31-9d21-462c221cd102 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:03.305412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7971150-4321-43de-9827-04d6c115ee7b None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:03.367079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791ffce-6870-47bb-978b-6db2203e60e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:03.368213 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791ffce-6870-47bb-978b-6db2203e60e7 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:03.368645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791ffce-6870-47bb-978b-6db2203e60e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:03.369120 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791ffce-6870-47bb-978b-6db2203e60e7 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:03.370396 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791ffce-6870-47bb-978b-6db2203e60e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:03.373301 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791ffce-6870-47bb-978b-6db2203e60e7 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:03.454332 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c5689-ed49-4e31-9d21-462c221cd102 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:03.529880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6a946-e7e1-41da-b99d-9b0b747a5397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:03.530974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6a946-e7e1-41da-b99d-9b0b747a5397 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:03.531514 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6a946-e7e1-41da-b99d-9b0b747a5397 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:03.532033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6a946-e7e1-41da-b99d-9b0b747a5397 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:03.534141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6a946-e7e1-41da-b99d-9b0b747a5397 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:03.534791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6a946-e7e1-41da-b99d-9b0b747a5397 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:03.537869 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6a946-e7e1-41da-b99d-9b0b747a5397 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:03.543660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 266/513] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:41:03 2022] POST /identity/v3/users => generated 247 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:41:03.547905 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791ffce-6870-47bb-978b-6db2203e60e7 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:03.572464 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:03.581842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 248/514] 10.222.0.94 () {56 vars in 1137 bytes} [Tue Jul 26 14:41:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:03.581842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6a946-e7e1-41da-b99d-9b0b747a5397 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:03.588166 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:03.758144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791ffce-6870-47bb-978b-6db2203e60e7 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:03.762228 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6a946-e7e1-41da-b99d-9b0b747a5397 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:03.816066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec8079-ad1e-448e-856e-29192f59502f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:03.817154 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec8079-ad1e-448e-856e-29192f59502f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:03.817594 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec8079-ad1e-448e-856e-29192f59502f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:03.818011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec8079-ad1e-448e-856e-29192f59502f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:03.835663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9749f-7982-4f27-b14b-3ef1a3aeb19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:03.837412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9749f-7982-4f27-b14b-3ef1a3aeb19c None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:03.837886 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9749f-7982-4f27-b14b-3ef1a3aeb19c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:03.838365 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9749f-7982-4f27-b14b-3ef1a3aeb19c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:03.843628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec8079-ad1e-448e-856e-29192f59502f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:03.847449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec8079-ad1e-448e-856e-29192f59502f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:03.859908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9749f-7982-4f27-b14b-3ef1a3aeb19c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:03.862563 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9749f-7982-4f27-b14b-3ef1a3aeb19c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-d6ec8079-ad1e-448e-856e-29192f59502f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:03.879445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:03.881497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:03.883917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:03.883917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:03.883917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:03.883917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:03.883917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 267/515] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:03.883917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec8079-ad1e-448e-856e-29192f59502f None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-dab9749f-7982-4f27-b14b-3ef1a3aeb19c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:03.899783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:03.901455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:03.903155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:03.903155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:03.903155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:03.903155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:03.903155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 249/516] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:03.904427 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9749f-7982-4f27-b14b-3ef1a3aeb19c None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:04.005190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:04.042408 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:04.084065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec8079-ad1e-448e-856e-29192f59502f None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:04.117901 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9749f-7982-4f27-b14b-3ef1a3aeb19c None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:04.159667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdefc1-fcd3-46db-b17a-0b8a5b8db95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:04.161144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdefc1-fcd3-46db-b17a-0b8a5b8db95e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:04.161658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdefc1-fcd3-46db-b17a-0b8a5b8db95e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:04.162164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdefc1-fcd3-46db-b17a-0b8a5b8db95e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:04.164025 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdefc1-fcd3-46db-b17a-0b8a5b8db95e None admin] RBAC: Adding query filter params (role.id=170d72cc35f34a249c65a819a18638a4, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=6a4278fe995e432d88f4346d6f54218b) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:04.164597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdefc1-fcd3-46db-b17a-0b8a5b8db95e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:04.167401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdefc1-fcd3-46db-b17a-0b8a5b8db95e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:04.179413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 268/517] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:03 2022] GET /identity/v3/role_assignments?user.id=6a4278fe995e432d88f4346d6f54218b&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=170d72cc35f34a249c65a819a18638a4 => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:04.181366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdefc1-fcd3-46db-b17a-0b8a5b8db95e None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:04.193060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120d5fa-99bf-4eb1-9d76-3fb90edf411f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:04.194719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120d5fa-99bf-4eb1-9d76-3fb90edf411f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:04.195449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120d5fa-99bf-4eb1-9d76-3fb90edf411f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:04.196235 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120d5fa-99bf-4eb1-9d76-3fb90edf411f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:04.354235 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120d5fa-99bf-4eb1-9d76-3fb90edf411f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:04.357272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120d5fa-99bf-4eb1-9d76-3fb90edf411f None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:04.378492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdefc1-fcd3-46db-b17a-0b8a5b8db95e None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:04.431806 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265aece-995e-494f-8489-fe9fad11db5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:04.432611 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265aece-995e-494f-8489-fe9fad11db5f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:04.432895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265aece-995e-494f-8489-fe9fad11db5f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:04.434147 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265aece-995e-494f-8489-fe9fad11db5f None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:04.435704 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:04.436766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265aece-995e-494f-8489-fe9fad11db5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:04.437198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265aece-995e-494f-8489-fe9fad11db5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:04.439351 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265aece-995e-494f-8489-fe9fad11db5f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:04.455247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 269/518] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:04.456890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265aece-995e-494f-8489-fe9fad11db5f None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:04.476220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:04.555575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 250/519] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:41:04.556750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120d5fa-99bf-4eb1-9d76-3fb90edf411f None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:04.660562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265aece-995e-494f-8489-fe9fad11db5f None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:04.735634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee00c6c-ff63-4fcc-9746-b6b0d8c36b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:04.736095 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120d5fa-99bf-4eb1-9d76-3fb90edf411f None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:04.736976 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee00c6c-ff63-4fcc-9746-b6b0d8c36b04 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:04.737283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee00c6c-ff63-4fcc-9746-b6b0d8c36b04 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:04.737523 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee00c6c-ff63-4fcc-9746-b6b0d8c36b04 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:04.738788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee00c6c-ff63-4fcc-9746-b6b0d8c36b04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:04.739501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee00c6c-ff63-4fcc-9746-b6b0d8c36b04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:04.742305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee00c6c-ff63-4fcc-9746-b6b0d8c36b04 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:04.766694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 270/520] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:04.768960 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee00c6c-ff63-4fcc-9746-b6b0d8c36b04 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:04.789730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927514e-27c4-4674-aede-11cbedd83098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:04.790419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927514e-27c4-4674-aede-11cbedd83098 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:04.790742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927514e-27c4-4674-aede-11cbedd83098 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:04.791188 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927514e-27c4-4674-aede-11cbedd83098 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:04.803490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927514e-27c4-4674-aede-11cbedd83098 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:04.805711 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927514e-27c4-4674-aede-11cbedd83098 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-1927514e-27c4-4674-aede-11cbedd83098 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:04.825928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:04.828820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:04.832227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:04.832227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:04.832227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:04.832227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:04.832227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:04.832227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:04.832227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:04.832227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:04.832227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 14:41:04.832227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:04.832227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 251/521] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:41:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:04.832227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927514e-27c4-4674-aede-11cbedd83098 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:04.862910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:04.888147 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:04.997903 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee00c6c-ff63-4fcc-9746-b6b0d8c36b04 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:05.010213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927514e-27c4-4674-aede-11cbedd83098 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:05.068060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996314e-c20f-485c-a597-71be692221ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:05.069305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996314e-c20f-485c-a597-71be692221ef None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:05.069892 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996314e-c20f-485c-a597-71be692221ef None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:05.070367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996314e-c20f-485c-a597-71be692221ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:05.072846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c26649-8b71-4e1a-9c19-b149738784e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:05.074314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c26649-8b71-4e1a-9c19-b149738784e1 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:05.074799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c26649-8b71-4e1a-9c19-b149738784e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:05.075277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c26649-8b71-4e1a-9c19-b149738784e1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:05.090540 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c26649-8b71-4e1a-9c19-b149738784e1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:05.093096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996314e-c20f-485c-a597-71be692221ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:05.093600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c26649-8b71-4e1a-9c19-b149738784e1 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:05.095015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996314e-c20f-485c-a597-71be692221ef None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-a5c26649-8b71-4e1a-9c19-b149738784e1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:05.112589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:05.113688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:05.114661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:05.114661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:05.114661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:05.114661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:05.114661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:05.114661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:05.114661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:05.114661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:05.114661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 14:41:05.114661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-b996314e-c20f-485c-a597-71be692221ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:05.116458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:05.117257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:05.118174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:05.118174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:05.118174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:05.118174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:05.121236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 252/522] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:05.122691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c26649-8b71-4e1a-9c19-b149738784e1 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:05.123153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 271/523] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:05.124313 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996314e-c20f-485c-a597-71be692221ef None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:05.244090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:05.290924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c26649-8b71-4e1a-9c19-b149738784e1 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:05.301930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:05.306399 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996314e-c20f-485c-a597-71be692221ef None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:05.364680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d242e2-a12c-4f6c-ba45-f76992a9f561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:05.365321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d242e2-a12c-4f6c-ba45-f76992a9f561 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:05.365603 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d242e2-a12c-4f6c-ba45-f76992a9f561 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:05.365969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d242e2-a12c-4f6c-ba45-f76992a9f561 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:05.366957 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d242e2-a12c-4f6c-ba45-f76992a9f561 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:05.367296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d242e2-a12c-4f6c-ba45-f76992a9f561 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:05.369045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d242e2-a12c-4f6c-ba45-f76992a9f561 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:05.369499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b2e70-03ac-450c-82bb-42d038ab5ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:05.371107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b2e70-03ac-450c-82bb-42d038ab5ed8 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:05.372378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b2e70-03ac-450c-82bb-42d038ab5ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:05.373177 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b2e70-03ac-450c-82bb-42d038ab5ed8 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:05.374751 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b2e70-03ac-450c-82bb-42d038ab5ed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:05.375715 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b2e70-03ac-450c-82bb-42d038ab5ed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:05.378314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b2e70-03ac-450c-82bb-42d038ab5ed8 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:05.391517 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 253/524] 10.222.0.94 () {56 vars in 1137 bytes} [Tue Jul 26 14:41:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:05.393706 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d242e2-a12c-4f6c-ba45-f76992a9f561 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:05.407759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 272/525] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:05.411455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b2e70-03ac-450c-82bb-42d038ab5ed8 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:05.556956 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d242e2-a12c-4f6c-ba45-f76992a9f561 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:05.583552 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b2e70-03ac-450c-82bb-42d038ab5ed8 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:05.605264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740e531-2393-43bc-b422-ab700034e182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:05.606177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740e531-2393-43bc-b422-ab700034e182 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:05.606529 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740e531-2393-43bc-b422-ab700034e182 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:05.606809 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740e531-2393-43bc-b422-ab700034e182 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:05.607868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740e531-2393-43bc-b422-ab700034e182 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:05.608133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740e531-2393-43bc-b422-ab700034e182 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:05.609680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740e531-2393-43bc-b422-ab700034e182 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:05.612344 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:05.635404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 254/526] 10.222.0.94 () {56 vars in 1139 bytes} [Tue Jul 26 14:41:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:05.636403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740e531-2393-43bc-b422-ab700034e182 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:05.657142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24102871-29a9-4daa-a6d4-81aaa43a3f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:05.659340 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24102871-29a9-4daa-a6d4-81aaa43a3f49 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:05.660189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24102871-29a9-4daa-a6d4-81aaa43a3f49 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:05.661240 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24102871-29a9-4daa-a6d4-81aaa43a3f49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:05.722740 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:05.771803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740e531-2393-43bc-b422-ab700034e182 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:05.809812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24102871-29a9-4daa-a6d4-81aaa43a3f49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:05.813346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24102871-29a9-4daa-a6d4-81aaa43a3f49 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:05.822619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e360c1-5a22-4838-a9f2-a4da76acf7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:05.823289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e360c1-5a22-4838-a9f2-a4da76acf7df None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:05.823637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e360c1-5a22-4838-a9f2-a4da76acf7df None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:05.824034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e360c1-5a22-4838-a9f2-a4da76acf7df None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:05.834743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e360c1-5a22-4838-a9f2-a4da76acf7df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:05.836752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e360c1-5a22-4838-a9f2-a4da76acf7df None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-83e360c1-5a22-4838-a9f2-a4da76acf7df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:05.851718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:05.852632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:05.853467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:05.853467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:05.853467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:05.853467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:05.853467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:05.853467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:05.853467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:05.853467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:05.853467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 255/527] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:05.853930 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e360c1-5a22-4838-a9f2-a4da76acf7df None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:05.927672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:05.966663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 273/528] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:41:05.968064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24102871-29a9-4daa-a6d4-81aaa43a3f49 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:05.990561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e360c1-5a22-4838-a9f2-a4da76acf7df None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:06.039143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4c005-ab25-4f07-9919-b831425b5c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:06.039872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4c005-ab25-4f07-9919-b831425b5c66 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:06.040159 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4c005-ab25-4f07-9919-b831425b5c66 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:06.040435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4c005-ab25-4f07-9919-b831425b5c66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:06.054446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:06.112662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24102871-29a9-4daa-a6d4-81aaa43a3f49 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:06.129932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4c005-ab25-4f07-9919-b831425b5c66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:06.131809 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4c005-ab25-4f07-9919-b831425b5c66 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:06.163225 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e07eee-1a80-4656-a976-1be08bc39eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:06.164426 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e07eee-1a80-4656-a976-1be08bc39eee None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:06.164926 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e07eee-1a80-4656-a976-1be08bc39eee None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:06.165426 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e07eee-1a80-4656-a976-1be08bc39eee None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:06.166595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e07eee-1a80-4656-a976-1be08bc39eee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:06.167287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e07eee-1a80-4656-a976-1be08bc39eee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:06.169206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e07eee-1a80-4656-a976-1be08bc39eee None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:06.195682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 274/529] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:06.196884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e07eee-1a80-4656-a976-1be08bc39eee None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:06.207388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:06.296480 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 256/530] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:41:06.298332 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4c005-ab25-4f07-9919-b831425b5c66 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:06.374409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e07eee-1a80-4656-a976-1be08bc39eee None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:06.416255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:06.444430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75009066-ccd6-4e3e-a62d-e38c58fe6aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:06.445649 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75009066-ccd6-4e3e-a62d-e38c58fe6aa6 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:06.446153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75009066-ccd6-4e3e-a62d-e38c58fe6aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:06.446625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75009066-ccd6-4e3e-a62d-e38c58fe6aa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:06.466900 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75009066-ccd6-4e3e-a62d-e38c58fe6aa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:06.468469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4c005-ab25-4f07-9919-b831425b5c66 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:06.469663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75009066-ccd6-4e3e-a62d-e38c58fe6aa6 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-75009066-ccd6-4e3e-a62d-e38c58fe6aa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:06.503077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:06.504033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:06.504989 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:06.504989 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:06.504989 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:06.504989 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:06.505424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 275/531] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:06.506973 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75009066-ccd6-4e3e-a62d-e38c58fe6aa6 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:06.521292 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdeb4f7-26f2-45d5-92ce-969dd5481188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:06.522535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeb4f7-26f2-45d5-92ce-969dd5481188 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:06.523094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeb4f7-26f2-45d5-92ce-969dd5481188 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:06.523633 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdeb4f7-26f2-45d5-92ce-969dd5481188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:06.525810 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdeb4f7-26f2-45d5-92ce-969dd5481188 None admin] RBAC: Adding query filter params (role.id=170d72cc35f34a249c65a819a18638a4, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=7448959048f746a0ae1588ff839deb0c) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:06.526440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdeb4f7-26f2-45d5-92ce-969dd5481188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:06.528352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdeb4f7-26f2-45d5-92ce-969dd5481188 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:06.538305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 257/532] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:06 2022] GET /identity/v3/role_assignments?user.id=7448959048f746a0ae1588ff839deb0c&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=170d72cc35f34a249c65a819a18638a4 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:06.539571 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdeb4f7-26f2-45d5-92ce-969dd5481188 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:06.567318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:06.730384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdeb4f7-26f2-45d5-92ce-969dd5481188 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:06.784871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eaa48d-c512-4431-b441-e2fac79f52a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:06.786297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eaa48d-c512-4431-b441-e2fac79f52a4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:06.786857 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eaa48d-c512-4431-b441-e2fac79f52a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:06.787338 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eaa48d-c512-4431-b441-e2fac79f52a4 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:06.788304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eaa48d-c512-4431-b441-e2fac79f52a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:06.788817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eaa48d-c512-4431-b441-e2fac79f52a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:06.791170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eaa48d-c512-4431-b441-e2fac79f52a4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:06.808654 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 258/533] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:06.810938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b75a11-59d5-43c1-83dc-430b477ccce4 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:06.811892 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b75a11-59d5-43c1-83dc-430b477ccce4 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:06.812486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b75a11-59d5-43c1-83dc-430b477ccce4 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:06.814264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 259/534] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:06.815748 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b75a11-59d5-43c1-83dc-430b477ccce4 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:06.854074 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75009066-ccd6-4e3e-a62d-e38c58fe6aa6 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:06.925018 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca50e8c-a028-4f59-a7fa-ffc315aa853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:06.925821 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca50e8c-a028-4f59-a7fa-ffc315aa853d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:06.926138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca50e8c-a028-4f59-a7fa-ffc315aa853d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:06.926481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca50e8c-a028-4f59-a7fa-ffc315aa853d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:06.946240 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca50e8c-a028-4f59-a7fa-ffc315aa853d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:06.947912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca50e8c-a028-4f59-a7fa-ffc315aa853d None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:06.964478 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-cca50e8c-a028-4f59-a7fa-ffc315aa853d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:06.966395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:06.968043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:06.969431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:06.969431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:06.969431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:06.969431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:06.974300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 276/535] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 462 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:06.975250 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffda739-db4f-4ca6-913a-0d5103284377 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:06.975589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffda739-db4f-4ca6-913a-0d5103284377 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:06.975871 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffda739-db4f-4ca6-913a-0d5103284377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:07.005348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-fffda739-db4f-4ca6-913a-0d5103284377 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:07.007250 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-fffda739-db4f-4ca6-913a-0d5103284377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:07.030221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b75a11-59d5-43c1-83dc-430b477ccce4 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:07.048510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:07.094602 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e8d90-7471-457e-a7a0-1348bf5548c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:07.096803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e8d90-7471-457e-a7a0-1348bf5548c7 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:07.097775 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e8d90-7471-457e-a7a0-1348bf5548c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:07.098517 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e8d90-7471-457e-a7a0-1348bf5548c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:07.118952 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e8d90-7471-457e-a7a0-1348bf5548c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:07.122918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e8d90-7471-457e-a7a0-1348bf5548c7 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:07.147628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 277/536] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-b73e8d90-7471-457e-a7a0-1348bf5548c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:07.148736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:07.150541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:07.151491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:07.151491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:07.151491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:07.151491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:07.151491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:07.151491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:07.151491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:07.151491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:07.151932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffda739-db4f-4ca6-913a-0d5103284377 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:07.155305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 260/537] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:07.163131 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e8d90-7471-457e-a7a0-1348bf5548c7 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:07.327246 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffda739-db4f-4ca6-913a-0d5103284377 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:07.349633 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e8d90-7471-457e-a7a0-1348bf5548c7 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:07.374467 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f35d92-c820-4378-afc5-adf054b22a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:07.375522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f35d92-c820-4378-afc5-adf054b22a69 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:07.375522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f35d92-c820-4378-afc5-adf054b22a69 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:07.375877 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f35d92-c820-4378-afc5-adf054b22a69 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:07.376603 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f35d92-c820-4378-afc5-adf054b22a69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:07.376919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f35d92-c820-4378-afc5-adf054b22a69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:07.378434 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f35d92-c820-4378-afc5-adf054b22a69 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:07.398101 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 278/538] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:07.398101 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f35d92-c820-4378-afc5-adf054b22a69 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:07.417583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00192a-fd3a-4e94-b588-a6d770388b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:07.417583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00192a-fd3a-4e94-b588-a6d770388b17 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:07.417583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00192a-fd3a-4e94-b588-a6d770388b17 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:07.418007 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00192a-fd3a-4e94-b588-a6d770388b17 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:07.419267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00192a-fd3a-4e94-b588-a6d770388b17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:07.419727 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00192a-fd3a-4e94-b588-a6d770388b17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:07.422314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00192a-fd3a-4e94-b588-a6d770388b17 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:07.428321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:07.431744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:07.450417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 261/539] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:07.450417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a51325-c57d-4c40-a6ab-88eecd62c99b None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:07.450417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a51325-c57d-4c40-a6ab-88eecd62c99b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:07.450417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a51325-c57d-4c40-a6ab-88eecd62c99b None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:07.450417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 262/540] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:07.451218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a51325-c57d-4c40-a6ab-88eecd62c99b None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:07.587304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a51325-c57d-4c40-a6ab-88eecd62c99b None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:07.592498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f35d92-c820-4378-afc5-adf054b22a69 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:07.645065 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b4747-377b-460e-8069-0f846a1a389e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:07.646416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b4747-377b-460e-8069-0f846a1a389e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:07.646772 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b4747-377b-460e-8069-0f846a1a389e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:07.647168 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b4747-377b-460e-8069-0f846a1a389e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:07.647376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece83d8-c67c-4a9a-b050-57d8454095d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:07.648438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece83d8-c67c-4a9a-b050-57d8454095d5 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:07.648844 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece83d8-c67c-4a9a-b050-57d8454095d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:07.649209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece83d8-c67c-4a9a-b050-57d8454095d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:07.661683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b4747-377b-460e-8069-0f846a1a389e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:07.663990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b4747-377b-460e-8069-0f846a1a389e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:07.669838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece83d8-c67c-4a9a-b050-57d8454095d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-aece83d8-c67c-4a9a-b050-57d8454095d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:07.682138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:07.683099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:07.684360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:07.684360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:07.684360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:07.684360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:07.684360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:07.684360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:07.684360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:07.684360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:07.684360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:07.684360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:07.684360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:07.684360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-c51b4747-377b-460e-8069-0f846a1a389e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:07.685060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:07.686631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:07.687535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:07.687535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:07.687535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:07.687535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:07.687535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:07.687535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:07.687535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:07.687535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:07.687958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 279/541] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:07.687958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece83d8-c67c-4a9a-b050-57d8454095d5 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:07.689541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 263/542] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:07.692438 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedc065-5516-48d1-ae3e-88f6fcce5b34 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:07.693000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedc065-5516-48d1-ae3e-88f6fcce5b34 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:07.693285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedc065-5516-48d1-ae3e-88f6fcce5b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:07.719008 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-bfedc065-5516-48d1-ae3e-88f6fcce5b34 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:07.720294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-bfedc065-5516-48d1-ae3e-88f6fcce5b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:07.797424 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:07.825434 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:07.831593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 264/543] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:07.831593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedc065-5516-48d1-ae3e-88f6fcce5b34 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:07.867984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece83d8-c67c-4a9a-b050-57d8454095d5 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:07.924079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2410e3a-f56c-44d2-b773-eb6998b3966a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:07.925659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2410e3a-f56c-44d2-b773-eb6998b3966a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:07.925959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2410e3a-f56c-44d2-b773-eb6998b3966a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:07.926668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2410e3a-f56c-44d2-b773-eb6998b3966a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:07.927798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2410e3a-f56c-44d2-b773-eb6998b3966a None admin] RBAC: Adding query filter params (role.id=170d72cc35f34a249c65a819a18638a4, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=b153935910c142aa80792e111872318a) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:07.928117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2410e3a-f56c-44d2-b773-eb6998b3966a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:07.930050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2410e3a-f56c-44d2-b773-eb6998b3966a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:07.941280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 280/544] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:07 2022] GET /identity/v3/role_assignments?user.id=b153935910c142aa80792e111872318a&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=170d72cc35f34a249c65a819a18638a4 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:07.943224 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2410e3a-f56c-44d2-b773-eb6998b3966a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:08.011130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedc065-5516-48d1-ae3e-88f6fcce5b34 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:08.083111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef016ea1-91c0-496c-a8a9-c086d2409c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:08.084842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef016ea1-91c0-496c-a8a9-c086d2409c35 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:08.085623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef016ea1-91c0-496c-a8a9-c086d2409c35 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:08.086383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef016ea1-91c0-496c-a8a9-c086d2409c35 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:08.088071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef016ea1-91c0-496c-a8a9-c086d2409c35 None admin] RBAC: Adding query filter params (name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:08.088907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef016ea1-91c0-496c-a8a9-c086d2409c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:08.091763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef016ea1-91c0-496c-a8a9-c086d2409c35 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:08.108438 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 265/545] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:08.110628 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef016ea1-91c0-496c-a8a9-c086d2409c35 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:08.187259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2410e3a-f56c-44d2-b773-eb6998b3966a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:08.240060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:08.281192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde40a2-dd61-491e-a093-db72cf9cac8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:08.282172 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde40a2-dd61-491e-a093-db72cf9cac8c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:08.282640 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde40a2-dd61-491e-a093-db72cf9cac8c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:08.283102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde40a2-dd61-491e-a093-db72cf9cac8c None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:08.285105 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde40a2-dd61-491e-a093-db72cf9cac8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:08.286643 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde40a2-dd61-491e-a093-db72cf9cac8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:08.289719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde40a2-dd61-491e-a093-db72cf9cac8c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:08.310140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:08.325149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 281/546] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:08.327503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde40a2-dd61-491e-a093-db72cf9cac8c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:08.352068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef016ea1-91c0-496c-a8a9-c086d2409c35 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:08.438583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035f133-b911-46f4-b0d3-18818cbea750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:08.439806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035f133-b911-46f4-b0d3-18818cbea750 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:08.440316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035f133-b911-46f4-b0d3-18818cbea750 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:08.440822 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035f133-b911-46f4-b0d3-18818cbea750 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:08.463170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035f133-b911-46f4-b0d3-18818cbea750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-c035f133-b911-46f4-b0d3-18818cbea750 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:08.475055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:08.476407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:08.477354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:08.477354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:08.477354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:08.477354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:08.477354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:08.477354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:08.477354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:08.477354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:08.477354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:08.477354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:08.477354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:08.477354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:08.483866 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 266/547] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:08.485655 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035f133-b911-46f4-b0d3-18818cbea750 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:08.575556 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde40a2-dd61-491e-a093-db72cf9cac8c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:08.665447 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19580f-04e8-4339-a511-19e6bd5dc505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:08.667683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19580f-04e8-4339-a511-19e6bd5dc505 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:08.668166 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19580f-04e8-4339-a511-19e6bd5dc505 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:08.668737 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19580f-04e8-4339-a511-19e6bd5dc505 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:08.696413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19580f-04e8-4339-a511-19e6bd5dc505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:08.700324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19580f-04e8-4339-a511-19e6bd5dc505 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:08.718412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035f133-b911-46f4-b0d3-18818cbea750 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-dd19580f-04e8-4339-a511-19e6bd5dc505 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:08.735707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:08.738513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:08.740823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:08.740823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:08.740823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:08.740823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:08.741331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:08.746004 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 282/548] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:08.746004 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19580f-04e8-4339-a511-19e6bd5dc505 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:08.793365 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83681e68-9afb-4366-9e9c-4192b7a3f38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:08.794418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83681e68-9afb-4366-9e9c-4192b7a3f38a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:08.794757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83681e68-9afb-4366-9e9c-4192b7a3f38a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:08.795137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83681e68-9afb-4366-9e9c-4192b7a3f38a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:08.796427 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83681e68-9afb-4366-9e9c-4192b7a3f38a None admin] RBAC: Adding query filter params (role.id=170d72cc35f34a249c65a819a18638a4, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=d39fd2708591424ca4892973e7da48ee) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:08.796840 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83681e68-9afb-4366-9e9c-4192b7a3f38a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:08.800166 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83681e68-9afb-4366-9e9c-4192b7a3f38a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:08.815617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 267/549] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:08 2022] GET /identity/v3/role_assignments?user.id=d39fd2708591424ca4892973e7da48ee&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=170d72cc35f34a249c65a819a18638a4 => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:08.815617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83681e68-9afb-4366-9e9c-4192b7a3f38a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:08.921752 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:09.007838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19580f-04e8-4339-a511-19e6bd5dc505 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:09.090323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7e0b3-2d4d-4e8e-9356-ac0a33d61ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:09.091365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7e0b3-2d4d-4e8e-9356-ac0a33d61ab3 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:09.091955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7e0b3-2d4d-4e8e-9356-ac0a33d61ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:09.092467 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7e0b3-2d4d-4e8e-9356-ac0a33d61ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:09.093666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c7e0b3-2d4d-4e8e-9356-ac0a33d61ab3 None admin] RBAC: Adding query filter params (name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:09.094292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c7e0b3-2d4d-4e8e-9356-ac0a33d61ab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:09.094416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83681e68-9afb-4366-9e9c-4192b7a3f38a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:09.096908 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c7e0b3-2d4d-4e8e-9356-ac0a33d61ab3 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:09.126822 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 283/550] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:09.136207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7e0b3-2d4d-4e8e-9356-ac0a33d61ab3 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:09.193123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2116e3-e98b-4699-95b7-0e0868fa0605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:09.194639 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2116e3-e98b-4699-95b7-0e0868fa0605 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:09.196634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2116e3-e98b-4699-95b7-0e0868fa0605 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:09.197417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2116e3-e98b-4699-95b7-0e0868fa0605 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:09.199108 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2116e3-e98b-4699-95b7-0e0868fa0605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:09.199920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2116e3-e98b-4699-95b7-0e0868fa0605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:09.203265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2116e3-e98b-4699-95b7-0e0868fa0605 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:09.280131 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 268/551] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:09.281966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2116e3-e98b-4699-95b7-0e0868fa0605 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:09.328244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c7e0b3-2d4d-4e8e-9356-ac0a33d61ab3 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:09.349159 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:09.354515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:09.391357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47cde1-1fcb-4be2-b978-0e4329f4cc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:09.393481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47cde1-1fcb-4be2-b978-0e4329f4cc52 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:09.394264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47cde1-1fcb-4be2-b978-0e4329f4cc52 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:09.394871 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47cde1-1fcb-4be2-b978-0e4329f4cc52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:09.413832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47cde1-1fcb-4be2-b978-0e4329f4cc52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:09.416730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47cde1-1fcb-4be2-b978-0e4329f4cc52 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-8f47cde1-1fcb-4be2-b978-0e4329f4cc52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:09.453570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:09.455052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:09.456380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:09.456380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:09.456380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:09.456380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:09.456380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 284/552] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:09.458772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab0ff3-e66b-4bca-ab40-377a3def5200 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:09.459261 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab0ff3-e66b-4bca-ab40-377a3def5200 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:09.459791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab0ff3-e66b-4bca-ab40-377a3def5200 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:09.462877 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 285/553] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:09.465113 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab0ff3-e66b-4bca-ab40-377a3def5200 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:09.491893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2116e3-e98b-4699-95b7-0e0868fa0605 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:09.555104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7dd4b-09e4-4d3f-90ee-18c75a00eec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:09.556193 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7dd4b-09e4-4d3f-90ee-18c75a00eec9 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:09.556709 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7dd4b-09e4-4d3f-90ee-18c75a00eec9 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:09.558075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7dd4b-09e4-4d3f-90ee-18c75a00eec9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:09.571774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7dd4b-09e4-4d3f-90ee-18c75a00eec9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:09.574227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7dd4b-09e4-4d3f-90ee-18c75a00eec9 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-7da7dd4b-09e4-4d3f-90ee-18c75a00eec9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:09.601340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:09.602351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:09.603386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:09.603386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:09.603386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:09.603386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:09.603386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:09.603386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:09.603386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:09.603386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:09.603386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:41:09.603386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:09.613115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 269/554] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:41:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:09.613115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25a6e9-3860-47e8-882d-b5195796049a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:09.613115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25a6e9-3860-47e8-882d-b5195796049a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:09.613115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25a6e9-3860-47e8-882d-b5195796049a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:09.646460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-5c25a6e9-3860-47e8-882d-b5195796049a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:09.648941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-5c25a6e9-3860-47e8-882d-b5195796049a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:09.692561 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab0ff3-e66b-4bca-ab40-377a3def5200 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:09.762339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26362f-a723-45ff-ac76-a38f1948c992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:09.763355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26362f-a723-45ff-ac76-a38f1948c992 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:09.763845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26362f-a723-45ff-ac76-a38f1948c992 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:09.764318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26362f-a723-45ff-ac76-a38f1948c992 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:09.765629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26362f-a723-45ff-ac76-a38f1948c992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:09.766144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26362f-a723-45ff-ac76-a38f1948c992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:09.769395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26362f-a723-45ff-ac76-a38f1948c992 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:09.770389 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 270/555] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:09.771700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25a6e9-3860-47e8-882d-b5195796049a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:09.795516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 286/556] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:41:09.806301 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26362f-a723-45ff-ac76-a38f1948c992 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:09.818213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:09.859810 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:09.948120 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25a6e9-3860-47e8-882d-b5195796049a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:10.005524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af5aae-7000-4cbf-b0ae-e4e59ef7ee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:10.006296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af5aae-7000-4cbf-b0ae-e4e59ef7ee8a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:10.006579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af5aae-7000-4cbf-b0ae-e4e59ef7ee8a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:10.006856 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af5aae-7000-4cbf-b0ae-e4e59ef7ee8a None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:10.007862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af5aae-7000-4cbf-b0ae-e4e59ef7ee8a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:10.008181 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af5aae-7000-4cbf-b0ae-e4e59ef7ee8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:10.009650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af5aae-7000-4cbf-b0ae-e4e59ef7ee8a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:10.014731 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26362f-a723-45ff-ac76-a38f1948c992 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:10.040933 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 271/557] 10.222.0.94 () {56 vars in 1135 bytes} [Tue Jul 26 14:41:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:10.041979 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af5aae-7000-4cbf-b0ae-e4e59ef7ee8a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:10.068558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030faf13-cc07-48c9-a75d-e9a49b37e98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:10.069780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030faf13-cc07-48c9-a75d-e9a49b37e98b None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:10.070476 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030faf13-cc07-48c9-a75d-e9a49b37e98b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:10.070998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030faf13-cc07-48c9-a75d-e9a49b37e98b None admin] PATH_INFO: `/v3/users/nova` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:10.081211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030faf13-cc07-48c9-a75d-e9a49b37e98b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:10.083409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030faf13-cc07-48c9-a75d-e9a49b37e98b None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-030faf13-cc07-48c9-a75d-e9a49b37e98b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:10.100245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:10.102545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:10.104137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:10.104137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:10.104137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:10.104137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:10.104137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:10.104137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:10.104137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:10.104137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:10.104137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:41:10.104137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:10.107895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 287/558] 10.222.0.94 () {56 vars in 1093 bytes} [Tue Jul 26 14:41:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:10.109321 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030faf13-cc07-48c9-a75d-e9a49b37e98b None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:10.148270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:10.173587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af5aae-7000-4cbf-b0ae-e4e59ef7ee8a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:10.223220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f1f91-27f4-4f35-84ba-35ed445b489f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:10.223499 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f1f91-27f4-4f35-84ba-35ed445b489f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:10.223723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f1f91-27f4-4f35-84ba-35ed445b489f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:10.223938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f1f91-27f4-4f35-84ba-35ed445b489f None admin] PATH_INFO: `/v3/roles/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:10.237788 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f1f91-27f4-4f35-84ba-35ed445b489f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:10.238124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-786f1f91-27f4-4f35-84ba-35ed445b489f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:10.244169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:10.245172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:10.246123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:10.246123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:10.246123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:10.246123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:10.246123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:10.246123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:10.246123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:10.246123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:10.246123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:10.246123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:10.246123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:10.246123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:10.252343 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 272/559] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:10.253388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f1f91-27f4-4f35-84ba-35ed445b489f None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:10.267483 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030faf13-cc07-48c9-a75d-e9a49b37e98b None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:10.331351 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee835a89-3d24-410d-8273-244c1de285d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:10.332378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee835a89-3d24-410d-8273-244c1de285d9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:10.333012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee835a89-3d24-410d-8273-244c1de285d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:10.333495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee835a89-3d24-410d-8273-244c1de285d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:10.387196 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f1f91-27f4-4f35-84ba-35ed445b489f None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:10.431080 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70342af-4532-4b5c-ad3a-7d6925f72583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:10.431772 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70342af-4532-4b5c-ad3a-7d6925f72583 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:10.432092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70342af-4532-4b5c-ad3a-7d6925f72583 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:10.432374 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70342af-4532-4b5c-ad3a-7d6925f72583 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:10.433211 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70342af-4532-4b5c-ad3a-7d6925f72583 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:10.433519 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70342af-4532-4b5c-ad3a-7d6925f72583 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:10.434987 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70342af-4532-4b5c-ad3a-7d6925f72583 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:10.446250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:10.458057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 273/560] 10.222.0.94 () {56 vars in 1133 bytes} [Tue Jul 26 14:41:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:10.459091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70342af-4532-4b5c-ad3a-7d6925f72583 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:10.467231 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee835a89-3d24-410d-8273-244c1de285d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:10.470323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee835a89-3d24-410d-8273-244c1de285d9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:10.623249 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:10.656262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70342af-4532-4b5c-ad3a-7d6925f72583 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:10.659289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 288/561] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:41:10.661439 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee835a89-3d24-410d-8273-244c1de285d9 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:10.739959 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e81ce-15c5-432c-9da7-72483769491c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:10.741035 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e81ce-15c5-432c-9da7-72483769491c None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:10.741534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e81ce-15c5-432c-9da7-72483769491c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:10.742029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e81ce-15c5-432c-9da7-72483769491c None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:10.743525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e81ce-15c5-432c-9da7-72483769491c None admin] RBAC: Adding query filter params (name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:10.744096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e81ce-15c5-432c-9da7-72483769491c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:10.746976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e81ce-15c5-432c-9da7-72483769491c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:10.772029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 274/562] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:10.773755 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e81ce-15c5-432c-9da7-72483769491c None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:10.858468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee835a89-3d24-410d-8273-244c1de285d9 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:10.912740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:10.942373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1db557-e784-4600-b718-e578f1ac2c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:10.943526 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1db557-e784-4600-b718-e578f1ac2c1e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:10.944032 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1db557-e784-4600-b718-e578f1ac2c1e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:10.944491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1db557-e784-4600-b718-e578f1ac2c1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:10.967077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1db557-e784-4600-b718-e578f1ac2c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:10.970085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1db557-e784-4600-b718-e578f1ac2c1e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:10.992242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e81ce-15c5-432c-9da7-72483769491c None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-ce1db557-e784-4600-b718-e578f1ac2c1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:11.003039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:11.005515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:11.007168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:11.007168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:11.007168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:11.007168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:11.012013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 289/563] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:11.013952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1db557-e784-4600-b718-e578f1ac2c1e None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:11.072527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f5bb4-74f2-4b7d-b955-2f7ab50478ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:11.073235 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f5bb4-74f2-4b7d-b955-2f7ab50478ae None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:11.073532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f5bb4-74f2-4b7d-b955-2f7ab50478ae None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:11.074300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f5bb4-74f2-4b7d-b955-2f7ab50478ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:11.088188 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:11.098355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f5bb4-74f2-4b7d-b955-2f7ab50478ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:11.100155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f5bb4-74f2-4b7d-b955-2f7ab50478ae None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-994f5bb4-74f2-4b7d-b955-2f7ab50478ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:11.124148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:11.125936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:11.129510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:11.129510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:11.129510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:11.129510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:11.129510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 275/564] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:11.129510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4b0f9-474b-4a43-9401-ce9da3e745c6 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:11.130813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4b0f9-474b-4a43-9401-ce9da3e745c6 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:11.130813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4b0f9-474b-4a43-9401-ce9da3e745c6 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:11.133103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 276/565] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:11.134602 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4b0f9-474b-4a43-9401-ce9da3e745c6 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:11.219521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1db557-e784-4600-b718-e578f1ac2c1e None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:11.300222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bf73f-f5ae-485e-821a-c61c0f62b853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:11.301358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bf73f-f5ae-485e-821a-c61c0f62b853 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:11.301867 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bf73f-f5ae-485e-821a-c61c0f62b853 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:11.302360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bf73f-f5ae-485e-821a-c61c0f62b853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:11.325213 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bf73f-f5ae-485e-821a-c61c0f62b853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:11.328150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bf73f-f5ae-485e-821a-c61c0f62b853 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:11.333300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4b0f9-474b-4a43-9401-ce9da3e745c6 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-417bf73f-f5ae-485e-821a-c61c0f62b853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:11.359673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:11.361411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:11.363318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:11.363318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:11.363318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:11.363318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:11.363318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 290/566] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:11.364815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bf73f-f5ae-485e-821a-c61c0f62b853 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:11.366051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:11.417999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9e981-33ce-4e1e-9abf-27d4cf4aef86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:11.419101 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9e981-33ce-4e1e-9abf-27d4cf4aef86 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:11.419601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9e981-33ce-4e1e-9abf-27d4cf4aef86 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:11.419987 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9e981-33ce-4e1e-9abf-27d4cf4aef86 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:11.421340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9e981-33ce-4e1e-9abf-27d4cf4aef86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:11.421920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9e981-33ce-4e1e-9abf-27d4cf4aef86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:11.425383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9e981-33ce-4e1e-9abf-27d4cf4aef86 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:11.451958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 277/567] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:11.457376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2e22c-cc5a-4a69-a531-3160ec43c46e None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:11.462867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2e22c-cc5a-4a69-a531-3160ec43c46e None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:11.463174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2e22c-cc5a-4a69-a531-3160ec43c46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:11.490559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-1bc2e22c-cc5a-4a69-a531-3160ec43c46e None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:11.492234 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1bc2e22c-cc5a-4a69-a531-3160ec43c46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:11.548313 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:11.574350 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bf73f-f5ae-485e-821a-c61c0f62b853 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:11.626570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 278/568] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:11.626570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2e22c-cc5a-4a69-a531-3160ec43c46e None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:11.643408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424634d-683e-4a5a-b193-c0c3269300d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:11.645217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424634d-683e-4a5a-b193-c0c3269300d8 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:11.645966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424634d-683e-4a5a-b193-c0c3269300d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:11.646889 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424634d-683e-4a5a-b193-c0c3269300d8 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:11.648552 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424634d-683e-4a5a-b193-c0c3269300d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:11.649389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424634d-683e-4a5a-b193-c0c3269300d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:11.652632 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424634d-683e-4a5a-b193-c0c3269300d8 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:11.675224 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 291/569] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:11.677110 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424634d-683e-4a5a-b193-c0c3269300d8 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:11.805585 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2e22c-cc5a-4a69-a531-3160ec43c46e None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:11.808998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:11.865118 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424634d-683e-4a5a-b193-c0c3269300d8 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:11.869515 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b272ef7-4d69-4b9a-a6b6-c2bba8080ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:11.870257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b272ef7-4d69-4b9a-a6b6-c2bba8080ab9 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:11.870724 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b272ef7-4d69-4b9a-a6b6-c2bba8080ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:11.871111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b272ef7-4d69-4b9a-a6b6-c2bba8080ab9 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:11.871954 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b272ef7-4d69-4b9a-a6b6-c2bba8080ab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:11.872272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b272ef7-4d69-4b9a-a6b6-c2bba8080ab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:11.874408 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b272ef7-4d69-4b9a-a6b6-c2bba8080ab9 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:11.897125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 279/570] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:11.898877 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b272ef7-4d69-4b9a-a6b6-c2bba8080ab9 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:11.929968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386003f2-a6ff-4b68-b62d-03b9d6d46992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:11.931909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386003f2-a6ff-4b68-b62d-03b9d6d46992 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:11.932429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386003f2-a6ff-4b68-b62d-03b9d6d46992 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:11.932882 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386003f2-a6ff-4b68-b62d-03b9d6d46992 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:11.947077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386003f2-a6ff-4b68-b62d-03b9d6d46992 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:11.949542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386003f2-a6ff-4b68-b62d-03b9d6d46992 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-386003f2-a6ff-4b68-b62d-03b9d6d46992 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:11.963745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:11.966065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:11.967825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:11.967825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:11.967825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:11.967825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:11.967825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:11.967825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:11.967825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:11.967825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:11.972232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 292/571] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:11.973387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386003f2-a6ff-4b68-b62d-03b9d6d46992 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:11.988161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:12.073696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b272ef7-4d69-4b9a-a6b6-c2bba8080ab9 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:12.117126 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386003f2-a6ff-4b68-b62d-03b9d6d46992 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:12.130261 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071857a5-0438-4850-b2bf-60d5d3a93e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:12.131215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071857a5-0438-4850-b2bf-60d5d3a93e74 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:12.131694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071857a5-0438-4850-b2bf-60d5d3a93e74 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:12.132139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071857a5-0438-4850-b2bf-60d5d3a93e74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:12.150392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071857a5-0438-4850-b2bf-60d5d3a93e74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-071857a5-0438-4850-b2bf-60d5d3a93e74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:12.160923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:12.172607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:12.174387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:12.174387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:12.174387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:12.174387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:12.174387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:12.174387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:12.174387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:12.174387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:12.174387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:12.174387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:12.174387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:12.174387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:12.174387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 280/572] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:12.174387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071857a5-0438-4850-b2bf-60d5d3a93e74 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:12.178519 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f437764-f1bd-4612-bd84-a0a62ae76b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:12.179784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f437764-f1bd-4612-bd84-a0a62ae76b88 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:12.180222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f437764-f1bd-4612-bd84-a0a62ae76b88 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:12.180650 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f437764-f1bd-4612-bd84-a0a62ae76b88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:12.192737 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f437764-f1bd-4612-bd84-a0a62ae76b88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:12.195264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f437764-f1bd-4612-bd84-a0a62ae76b88 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:12.215847 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-6f437764-f1bd-4612-bd84-a0a62ae76b88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:12.216136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:12.218180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:12.219721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:12.219721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:12.219721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:12.219721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:12.219721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:12.219721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:12.219721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:12.219721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:12.219721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 293/573] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:12.221352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e77c24-f002-4986-865e-e1259805fb37 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:12.221352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e77c24-f002-4986-865e-e1259805fb37 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:12.221880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e77c24-f002-4986-865e-e1259805fb37 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:12.227759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 294/574] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:12.229583 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e77c24-f002-4986-865e-e1259805fb37 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:12.341996 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071857a5-0438-4850-b2bf-60d5d3a93e74 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:12.356478 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:12.399050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3e805-1296-4291-b4f1-ce506f4194c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:12.400133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3e805-1296-4291-b4f1-ce506f4194c7 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:12.400586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3e805-1296-4291-b4f1-ce506f4194c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:12.401018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3e805-1296-4291-b4f1-ce506f4194c7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:12.413186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3e805-1296-4291-b4f1-ce506f4194c7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:12.414889 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3e805-1296-4291-b4f1-ce506f4194c7 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:12.424470 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e77c24-f002-4986-865e-e1259805fb37 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-5cc3e805-1296-4291-b4f1-ce506f4194c7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:12.434687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:12.437456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:12.439300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:12.439300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:12.439300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:12.439300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:12.439300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:12.439300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:12.439300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:12.439300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:12.439300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:41:12.439300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:12.439300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 281/575] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:41:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:12.439300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3e805-1296-4291-b4f1-ce506f4194c7 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:12.499823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b163f-ad0f-41c1-92dd-16ef11fbf737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:12.500896 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b163f-ad0f-41c1-92dd-16ef11fbf737 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:12.501408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b163f-ad0f-41c1-92dd-16ef11fbf737 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:12.501943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b163f-ad0f-41c1-92dd-16ef11fbf737 None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:12.503423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b163f-ad0f-41c1-92dd-16ef11fbf737 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:12.504003 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b163f-ad0f-41c1-92dd-16ef11fbf737 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:12.506832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b163f-ad0f-41c1-92dd-16ef11fbf737 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:12.531552 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 295/576] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:12.533681 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b163f-ad0f-41c1-92dd-16ef11fbf737 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:12.588747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:12.613227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3e805-1296-4291-b4f1-ce506f4194c7 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:12.672303 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c1edb-edc3-46b9-9f29-885d40e7c290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:12.672947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c1edb-edc3-46b9-9f29-885d40e7c290 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:12.673231 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c1edb-edc3-46b9-9f29-885d40e7c290 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:12.674602 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c1edb-edc3-46b9-9f29-885d40e7c290 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:12.675508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c1edb-edc3-46b9-9f29-885d40e7c290 None admin] RBAC: Adding query filter params (name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:12.675839 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c1edb-edc3-46b9-9f29-885d40e7c290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:12.677642 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c1edb-edc3-46b9-9f29-885d40e7c290 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:12.698770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 282/577] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:12.698770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af792b1-5a12-488f-8090-ffebb7fbb8cb None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:12.698770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af792b1-5a12-488f-8090-ffebb7fbb8cb None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:12.698770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af792b1-5a12-488f-8090-ffebb7fbb8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:12.721611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-3af792b1-5a12-488f-8090-ffebb7fbb8cb None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:12.722947 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b163f-ad0f-41c1-92dd-16ef11fbf737 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:12.723103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3af792b1-5a12-488f-8090-ffebb7fbb8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:12.762343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:12.773476 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034046b-6a09-4c7a-8419-190def304bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:12.774113 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034046b-6a09-4c7a-8419-190def304bf7 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:12.774495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034046b-6a09-4c7a-8419-190def304bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:12.774964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034046b-6a09-4c7a-8419-190def304bf7 None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:12.778731 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034046b-6a09-4c7a-8419-190def304bf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:12.778731 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034046b-6a09-4c7a-8419-190def304bf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:12.779588 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034046b-6a09-4c7a-8419-190def304bf7 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:12.804582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 296/578] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:12.806224 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034046b-6a09-4c7a-8419-190def304bf7 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:12.822550 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 283/579] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:12.823683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af792b1-5a12-488f-8090-ffebb7fbb8cb None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:12.919366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:12.951335 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af792b1-5a12-488f-8090-ffebb7fbb8cb None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:12.990678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034046b-6a09-4c7a-8419-190def304bf7 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:12.996928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcba89e-5166-49c8-851b-37a8e91d16d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:12.997664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcba89e-5166-49c8-851b-37a8e91d16d7 None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:12.997946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcba89e-5166-49c8-851b-37a8e91d16d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:12.998261 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcba89e-5166-49c8-851b-37a8e91d16d7 None admin] PATH_INFO: `/v3/projects/a9b328041da24cedbc801b271d6e0419/users/6a4278fe995e432d88f4346d6f54218b/roles/170d72cc35f34a249c65a819a18638a4` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:13.025279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcba89e-5166-49c8-851b-37a8e91d16d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9b328041da24cedbc801b271d6e0419, user_id=6a4278fe995e432d88f4346d6f54218b, role_id=170d72cc35f34a249c65a819a18638a4)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:13.027662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcba89e-5166-49c8-851b-37a8e91d16d7 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:13.037079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2e9d6-b2f0-4690-921c-428812e47f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:13.037868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2e9d6-b2f0-4690-921c-428812e47f06 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:13.038232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2e9d6-b2f0-4690-921c-428812e47f06 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:13.038588 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2e9d6-b2f0-4690-921c-428812e47f06 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:13.039599 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2e9d6-b2f0-4690-921c-428812e47f06 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:13.040002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2e9d6-b2f0-4690-921c-428812e47f06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:13.041553 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2e9d6-b2f0-4690-921c-428812e47f06 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:13.111269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 297/580] 10.222.0.94 () {56 vars in 1137 bytes} [Tue Jul 26 14:41:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:13.111269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2e9d6-b2f0-4690-921c-428812e47f06 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:13.123728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 284/581] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:41:12 2022] PUT /identity/v3/projects/a9b328041da24cedbc801b271d6e0419/users/6a4278fe995e432d88f4346d6f54218b/roles/170d72cc35f34a249c65a819a18638a4 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:41:13.125496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcba89e-5166-49c8-851b-37a8e91d16d7 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:13.197146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:13.289039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2e9d6-b2f0-4690-921c-428812e47f06 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:13.347155 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edecf4d-0462-4014-b1ba-c9ba516ab3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:13.348050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edecf4d-0462-4014-b1ba-c9ba516ab3d3 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:13.348439 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edecf4d-0462-4014-b1ba-c9ba516ab3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:13.348684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edecf4d-0462-4014-b1ba-c9ba516ab3d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:13.370058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edecf4d-0462-4014-b1ba-c9ba516ab3d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:13.372713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcba89e-5166-49c8-851b-37a8e91d16d7 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:13.372930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edecf4d-0462-4014-b1ba-c9ba516ab3d3 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-8edecf4d-0462-4014-b1ba-c9ba516ab3d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:13.403850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:13.405336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:13.407026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:13.407026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:13.407026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:13.407026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:13.407026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 298/582] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:13.407656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edecf4d-0462-4014-b1ba-c9ba516ab3d3 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:13.419878 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:13.435305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1feb0-871f-43c1-90cf-27c63c886352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:13.436393 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1feb0-871f-43c1-90cf-27c63c886352 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:13.436782 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1feb0-871f-43c1-90cf-27c63c886352 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:13.437251 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1feb0-871f-43c1-90cf-27c63c886352 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:13.438285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1feb0-871f-43c1-90cf-27c63c886352 None admin] RBAC: Adding query filter params (role.id=170d72cc35f34a249c65a819a18638a4, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=b6cea7b5ba4a4acabe8a26837cd97310) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:13.438729 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1feb0-871f-43c1-90cf-27c63c886352 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:13.440694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1feb0-871f-43c1-90cf-27c63c886352 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:13.455610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 285/583] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:13 2022] GET /identity/v3/role_assignments?user.id=b6cea7b5ba4a4acabe8a26837cd97310&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=170d72cc35f34a249c65a819a18638a4 => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:13.455610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1feb0-871f-43c1-90cf-27c63c886352 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:13.589481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edecf4d-0462-4014-b1ba-c9ba516ab3d3 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:13.603902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:13.640687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9957af-c002-479f-b45a-1c0320ddd56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:13.640687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9957af-c002-479f-b45a-1c0320ddd56a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:13.640687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9957af-c002-479f-b45a-1c0320ddd56a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:13.641088 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9957af-c002-479f-b45a-1c0320ddd56a None admin] PATH_INFO: `/v3/roles/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:13.656331 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9957af-c002-479f-b45a-1c0320ddd56a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-1d9957af-c002-479f-b45a-1c0320ddd56a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:13.663303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:13.664484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:13.665504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:13.665504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:13.665504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:13.665504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:13.665504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:13.665504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:13.665504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:13.665504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:13.665504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:13.665504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:13.665504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:13.665504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:13.665504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 299/584] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:13.667364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9957af-c002-479f-b45a-1c0320ddd56a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:13.687893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1feb0-871f-43c1-90cf-27c63c886352 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:13.758189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e122894-cc3f-43a4-a9b0-c3d248acb780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:13.759348 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e122894-cc3f-43a4-a9b0-c3d248acb780 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:13.759875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e122894-cc3f-43a4-a9b0-c3d248acb780 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:13.760367 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e122894-cc3f-43a4-a9b0-c3d248acb780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:13.783170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e122894-cc3f-43a4-a9b0-c3d248acb780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:13.787055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e122894-cc3f-43a4-a9b0-c3d248acb780 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-6e122894-cc3f-43a4-a9b0-c3d248acb780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:13.818824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:13.819826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:13.820703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:13.820703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:13.820703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:13.820703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:13.821195 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 286/585] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:13.827611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e122894-cc3f-43a4-a9b0-c3d248acb780 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:13.880076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9957af-c002-479f-b45a-1c0320ddd56a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:13.905889 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:13.951946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316dda9-4ab8-4af1-852e-8299124cf198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:13.953619 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316dda9-4ab8-4af1-852e-8299124cf198 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:13.954405 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316dda9-4ab8-4af1-852e-8299124cf198 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:13.955905 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316dda9-4ab8-4af1-852e-8299124cf198 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:13.957716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316dda9-4ab8-4af1-852e-8299124cf198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:13.958610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316dda9-4ab8-4af1-852e-8299124cf198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:13.961595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316dda9-4ab8-4af1-852e-8299124cf198 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:13.987902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 300/586] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:13.990107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316dda9-4ab8-4af1-852e-8299124cf198 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:14.005141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e122894-cc3f-43a4-a9b0-c3d248acb780 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:14.033205 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:14.055533 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b0403-3db1-4ca4-b605-8b63ce3f99de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:14.056425 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b0403-3db1-4ca4-b605-8b63ce3f99de None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:14.057027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b0403-3db1-4ca4-b605-8b63ce3f99de None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:14.057489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b0403-3db1-4ca4-b605-8b63ce3f99de None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:14.058504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b0403-3db1-4ca4-b605-8b63ce3f99de None admin] RBAC: Adding query filter params (name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:14.058984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b0403-3db1-4ca4-b605-8b63ce3f99de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:14.060697 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b0403-3db1-4ca4-b605-8b63ce3f99de None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:14.071495 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 287/587] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:14.072603 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b0403-3db1-4ca4-b605-8b63ce3f99de None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:14.198870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b0403-3db1-4ca4-b605-8b63ce3f99de None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:14.201343 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:14.204721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316dda9-4ab8-4af1-852e-8299124cf198 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:14.247382 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce83bb-7bb8-4996-9e45-50cc08c1c2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:14.248672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce83bb-7bb8-4996-9e45-50cc08c1c2a9 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:14.249129 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce83bb-7bb8-4996-9e45-50cc08c1c2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:14.249582 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce83bb-7bb8-4996-9e45-50cc08c1c2a9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:14.261575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce83bb-7bb8-4996-9e45-50cc08c1c2a9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:14.263677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce83bb-7bb8-4996-9e45-50cc08c1c2a9 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-60ce83bb-7bb8-4996-9e45-50cc08c1c2a9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:14.279376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:14.289758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:14.290896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:14.290896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:14.290896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:14.290896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:14.290896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:14.290896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:14.290896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:14.290896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:14.290896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 14:41:14.290896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:14.291424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a9e5f-b310-49f3-abb8-27a0947f3b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:14.291547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 288/588] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:41:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:14.291547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce83bb-7bb8-4996-9e45-50cc08c1c2a9 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:14.291695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a9e5f-b310-49f3-abb8-27a0947f3b34 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:14.291695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a9e5f-b310-49f3-abb8-27a0947f3b34 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:14.291695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a9e5f-b310-49f3-abb8-27a0947f3b34 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:14.291695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a9e5f-b310-49f3-abb8-27a0947f3b34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:14.292065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a9e5f-b310-49f3-abb8-27a0947f3b34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:14.294480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a9e5f-b310-49f3-abb8-27a0947f3b34 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:14.321660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 301/589] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:14.323797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a9e5f-b310-49f3-abb8-27a0947f3b34 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:14.423750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce83bb-7bb8-4996-9e45-50cc08c1c2a9 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:14.465408 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94fd52-59b7-4f72-807b-c639561cb6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:14.466985 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94fd52-59b7-4f72-807b-c639561cb6d0 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:14.467451 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94fd52-59b7-4f72-807b-c639561cb6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:14.468024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94fd52-59b7-4f72-807b-c639561cb6d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:14.482858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94fd52-59b7-4f72-807b-c639561cb6d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:14.485074 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94fd52-59b7-4f72-807b-c639561cb6d0 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:14.489251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-cf94fd52-59b7-4f72-807b-c639561cb6d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:14.505647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:14.506835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:14.507924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:14.507924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:14.507924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:14.507924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:14.509990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 289/590] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:14.511172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94fd52-59b7-4f72-807b-c639561cb6d0 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:14.531667 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:14.533588 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a9e5f-b310-49f3-abb8-27a0947f3b34 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:14.604273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff317496-3155-4a70-80f7-357707771330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:14.605219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff317496-3155-4a70-80f7-357707771330 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:14.605694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff317496-3155-4a70-80f7-357707771330 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:14.606165 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff317496-3155-4a70-80f7-357707771330 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:14.607815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff317496-3155-4a70-80f7-357707771330 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:14.608355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff317496-3155-4a70-80f7-357707771330 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:14.611914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff317496-3155-4a70-80f7-357707771330 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:14.640469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94fd52-59b7-4f72-807b-c639561cb6d0 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:14.655879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 302/591] 10.222.0.94 () {56 vars in 1137 bytes} [Tue Jul 26 14:41:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:14.657810 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff317496-3155-4a70-80f7-357707771330 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:14.682809 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbbe45-4e75-40cc-894e-e4d94f66eba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:14.684245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbbe45-4e75-40cc-894e-e4d94f66eba3 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:14.684790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbbe45-4e75-40cc-894e-e4d94f66eba3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:14.685252 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbbe45-4e75-40cc-894e-e4d94f66eba3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:14.694158 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbbe45-4e75-40cc-894e-e4d94f66eba3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:14.696133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbbe45-4e75-40cc-894e-e4d94f66eba3 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-d2dbbe45-4e75-40cc-894e-e4d94f66eba3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:14.710103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:14.711275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:14.712331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:14.712331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:14.712331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:14.712331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:14.712331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:14.712331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:14.712331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:14.712331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:14.714635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 290/592] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:14.715869 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbbe45-4e75-40cc-894e-e4d94f66eba3 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:14.809741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:14.842268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbbe45-4e75-40cc-894e-e4d94f66eba3 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:14.866892 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff317496-3155-4a70-80f7-357707771330 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:14.889713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3fc7-11c1-41c9-9fc4-e5c910e33c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:14.890720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3fc7-11c1-41c9-9fc4-e5c910e33c04 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:14.891321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3fc7-11c1-41c9-9fc4-e5c910e33c04 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:14.891811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3fc7-11c1-41c9-9fc4-e5c910e33c04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:14.906223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c3fc7-11c1-41c9-9fc4-e5c910e33c04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:14.908720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c3fc7-11c1-41c9-9fc4-e5c910e33c04 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-8b0c3fc7-11c1-41c9-9fc4-e5c910e33c04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:14.931707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:14.933097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:14.934010 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:14.934010 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:14.934010 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:14.934010 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:14.937532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 291/593] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:14.939005 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3fc7-11c1-41c9-9fc4-e5c910e33c04 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:14.948876 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c110c7-183b-4f35-bb39-5dba62531f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:14.949914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c110c7-183b-4f35-bb39-5dba62531f80 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:14.950394 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c110c7-183b-4f35-bb39-5dba62531f80 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:14.950868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c110c7-183b-4f35-bb39-5dba62531f80 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:14.952257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c110c7-183b-4f35-bb39-5dba62531f80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:14.953251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c110c7-183b-4f35-bb39-5dba62531f80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:14.956238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c110c7-183b-4f35-bb39-5dba62531f80 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:14.961548 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:14.984073 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 303/594] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:14.986035 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c110c7-183b-4f35-bb39-5dba62531f80 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:15.143985 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3fc7-11c1-41c9-9fc4-e5c910e33c04 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:15.192041 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:15.201913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c110c7-183b-4f35-bb39-5dba62531f80 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:15.202981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994c46b-c120-40b0-b1da-bd8433d72be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:15.204206 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c46b-c120-40b0-b1da-bd8433d72be4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:15.204858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c46b-c120-40b0-b1da-bd8433d72be4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:15.205360 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c46b-c120-40b0-b1da-bd8433d72be4 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:15.206790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994c46b-c120-40b0-b1da-bd8433d72be4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:15.207410 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994c46b-c120-40b0-b1da-bd8433d72be4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:15.209745 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994c46b-c120-40b0-b1da-bd8433d72be4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:15.228124 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 292/595] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:15.229400 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994c46b-c120-40b0-b1da-bd8433d72be4 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:15.284741 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97a8f4-2e62-45fd-8bb3-6c8fa334f720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:15.286198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97a8f4-2e62-45fd-8bb3-6c8fa334f720 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:15.286655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97a8f4-2e62-45fd-8bb3-6c8fa334f720 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:15.287099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97a8f4-2e62-45fd-8bb3-6c8fa334f720 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:15.288415 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97a8f4-2e62-45fd-8bb3-6c8fa334f720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:15.288919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97a8f4-2e62-45fd-8bb3-6c8fa334f720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:15.291647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97a8f4-2e62-45fd-8bb3-6c8fa334f720 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:15.324348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 304/596] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:15.325973 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97a8f4-2e62-45fd-8bb3-6c8fa334f720 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:15.386645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994c46b-c120-40b0-b1da-bd8433d72be4 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:15.431858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1dabb-211e-4446-bed4-c65a425e7c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:15.432967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1dabb-211e-4446-bed4-c65a425e7c65 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:15.433761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1dabb-211e-4446-bed4-c65a425e7c65 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:15.434323 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1dabb-211e-4446-bed4-c65a425e7c65 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:15.444270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1dabb-211e-4446-bed4-c65a425e7c65 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:15.446912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1dabb-211e-4446-bed4-c65a425e7c65 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:15.456500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-9dd1dabb-211e-4446-bed4-c65a425e7c65 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:15.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:15.466916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:15.468418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:15.468418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:15.468418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:15.468418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:15.468418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:15.468418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:15.468418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:15.468418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:15.468418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 14:41:15.468418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:15.476574 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 293/597] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:15.476574 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1dabb-211e-4446-bed4-c65a425e7c65 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:15.546132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97a8f4-2e62-45fd-8bb3-6c8fa334f720 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:15.558867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:15.615949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d8ff7-be44-4e31-b237-dab4b8f24306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:15.617017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d8ff7-be44-4e31-b237-dab4b8f24306 None admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:15.617483 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d8ff7-be44-4e31-b237-dab4b8f24306 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:15.618120 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d8ff7-be44-4e31-b237-dab4b8f24306 None admin] PATH_INFO: `/v3/projects/a9b328041da24cedbc801b271d6e0419/users/7448959048f746a0ae1588ff839deb0c/roles/170d72cc35f34a249c65a819a18638a4` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:15.648856 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d8ff7-be44-4e31-b237-dab4b8f24306 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9b328041da24cedbc801b271d6e0419, user_id=7448959048f746a0ae1588ff839deb0c, role_id=170d72cc35f34a249c65a819a18638a4)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:15.651727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d8ff7-be44-4e31-b237-dab4b8f24306 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:15.663286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1dabb-211e-4446-bed4-c65a425e7c65 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:15.736857 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f930c-8b29-4ab9-a60f-6912a2c3316b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:15.737735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f930c-8b29-4ab9-a60f-6912a2c3316b None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:15.738156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f930c-8b29-4ab9-a60f-6912a2c3316b None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:15.738563 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f930c-8b29-4ab9-a60f-6912a2c3316b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:15.751187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f930c-8b29-4ab9-a60f-6912a2c3316b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:15.752251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 305/598] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:41:15 2022] PUT /identity/v3/projects/a9b328041da24cedbc801b271d6e0419/users/7448959048f746a0ae1588ff839deb0c/roles/170d72cc35f34a249c65a819a18638a4 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:41:15.754218 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d8ff7-be44-4e31-b237-dab4b8f24306 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:15.754443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f930c-8b29-4ab9-a60f-6912a2c3316b None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-118f930c-8b29-4ab9-a60f-6912a2c3316b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:15.796135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:15.797932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:15.800371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:15.800371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:15.800371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:15.800371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:15.800371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:15.800371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:15.800371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:15.800371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:15.807513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 294/599] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:15.809242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f930c-8b29-4ab9-a60f-6912a2c3316b None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:16.024257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d8ff7-be44-4e31-b237-dab4b8f24306 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:16.043878 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:16.045368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f930c-8b29-4ab9-a60f-6912a2c3316b None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:16.056426 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:16.094988 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014ffeb-67a8-4729-a5bb-fb993cf3f2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:16.096291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014ffeb-67a8-4729-a5bb-fb993cf3f2ad None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:16.098212 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014ffeb-67a8-4729-a5bb-fb993cf3f2ad None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:16.098517 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014ffeb-67a8-4729-a5bb-fb993cf3f2ad None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:16.100434 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014ffeb-67a8-4729-a5bb-fb993cf3f2ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:16.100884 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014ffeb-67a8-4729-a5bb-fb993cf3f2ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:16.102691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014ffeb-67a8-4729-a5bb-fb993cf3f2ad None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:16.107446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f6235-5ff8-4621-8ee6-707cf157b141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:16.107446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f6235-5ff8-4621-8ee6-707cf157b141 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:16.107446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f6235-5ff8-4621-8ee6-707cf157b141 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:16.108145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f6235-5ff8-4621-8ee6-707cf157b141 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:16.109672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f6235-5ff8-4621-8ee6-707cf157b141 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:16.110514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f6235-5ff8-4621-8ee6-707cf157b141 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:16.115280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f6235-5ff8-4621-8ee6-707cf157b141 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:16.120655 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 295/600] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:16.131894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014ffeb-67a8-4729-a5bb-fb993cf3f2ad None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:16.163533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 306/601] 10.222.0.94 () {56 vars in 1139 bytes} [Tue Jul 26 14:41:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:16.168121 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f6235-5ff8-4621-8ee6-707cf157b141 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:16.458814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014ffeb-67a8-4729-a5bb-fb993cf3f2ad None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:16.484136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f6235-5ff8-4621-8ee6-707cf157b141 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:16.526841 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c3c05-da1e-4444-a5c6-5dbd6c997d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:16.528993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c3c05-da1e-4444-a5c6-5dbd6c997d90 None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:16.529685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c3c05-da1e-4444-a5c6-5dbd6c997d90 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:16.530357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c3c05-da1e-4444-a5c6-5dbd6c997d90 None admin] PATH_INFO: `/v3/projects/a9b328041da24cedbc801b271d6e0419/users/b153935910c142aa80792e111872318a/roles/170d72cc35f34a249c65a819a18638a4` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:16.553608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6929ea-9d74-4c42-bbe5-142ff43dcb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:16.554831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6929ea-9d74-4c42-bbe5-142ff43dcb04 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:16.555939 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6929ea-9d74-4c42-bbe5-142ff43dcb04 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:16.555939 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6929ea-9d74-4c42-bbe5-142ff43dcb04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:16.561246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c3c05-da1e-4444-a5c6-5dbd6c997d90 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9b328041da24cedbc801b271d6e0419, user_id=b153935910c142aa80792e111872318a, role_id=170d72cc35f34a249c65a819a18638a4)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:16.564979 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c3c05-da1e-4444-a5c6-5dbd6c997d90 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:16.576115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6929ea-9d74-4c42-bbe5-142ff43dcb04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:16.577961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6929ea-9d74-4c42-bbe5-142ff43dcb04 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-5b6929ea-9d74-4c42-bbe5-142ff43dcb04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:16.601007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:16.602060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:16.603291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:16.603291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:16.603291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:16.603291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:16.603291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 307/602] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:16.604371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a26ccd-6199-4df8-bd71-09171e753eea None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:16.604703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a26ccd-6199-4df8-bd71-09171e753eea None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:16.605107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a26ccd-6199-4df8-bd71-09171e753eea None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:16.606832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 308/603] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:16.609690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a26ccd-6199-4df8-bd71-09171e753eea None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:16.635996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:16.721125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 296/604] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:41:16 2022] PUT /identity/v3/projects/a9b328041da24cedbc801b271d6e0419/users/b153935910c142aa80792e111872318a/roles/170d72cc35f34a249c65a819a18638a4 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:41:16.725111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29947835-aacd-4ca7-b2a5-f07a23464026 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:16.725809 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29947835-aacd-4ca7-b2a5-f07a23464026 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:16.726608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29947835-aacd-4ca7-b2a5-f07a23464026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:16.765157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-29947835-aacd-4ca7-b2a5-f07a23464026 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:16.766593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-29947835-aacd-4ca7-b2a5-f07a23464026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:16.784296 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a26ccd-6199-4df8-bd71-09171e753eea None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:16.784835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:16.840649 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d52fa-d2a4-47b4-8ed0-68361348e70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:16.841744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d52fa-d2a4-47b4-8ed0-68361348e70d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:16.842245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d52fa-d2a4-47b4-8ed0-68361348e70d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:16.842723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d52fa-d2a4-47b4-8ed0-68361348e70d None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:16.843850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d52fa-d2a4-47b4-8ed0-68361348e70d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:16.844532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d52fa-d2a4-47b4-8ed0-68361348e70d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:16.847492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d52fa-d2a4-47b4-8ed0-68361348e70d None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:16.875269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 309/605] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:16.877542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4f5c5-0e7b-4ee5-b680-3b4e5a928c0e None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:16.878493 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4f5c5-0e7b-4ee5-b680-3b4e5a928c0e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:16.879016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4f5c5-0e7b-4ee5-b680-3b4e5a928c0e None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:16.881616 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 310/606] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:16.882824 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4f5c5-0e7b-4ee5-b680-3b4e5a928c0e None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:16.887719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 297/607] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:16.898254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bb00f-d4c1-4c3c-a813-49781afe6b8b None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:16.899220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bb00f-d4c1-4c3c-a813-49781afe6b8b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:16.900252 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bb00f-d4c1-4c3c-a813-49781afe6b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:16.971762 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-177bb00f-d4c1-4c3c-a813-49781afe6b8b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:16.974580 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-177bb00f-d4c1-4c3c-a813-49781afe6b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:17.009178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:17.054788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4f5c5-0e7b-4ee5-b680-3b4e5a928c0e None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:17.103611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 298/608] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:17.105363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bb00f-d4c1-4c3c-a813-49781afe6b8b None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:17.140079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d414f-5f63-438a-9ef7-448dc225adb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:17.140830 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d414f-5f63-438a-9ef7-448dc225adb1 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:17.141104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d414f-5f63-438a-9ef7-448dc225adb1 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:17.141361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d414f-5f63-438a-9ef7-448dc225adb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:17.150377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d414f-5f63-438a-9ef7-448dc225adb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:17.151922 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d414f-5f63-438a-9ef7-448dc225adb1 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-5c4d414f-5f63-438a-9ef7-448dc225adb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:17.167013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:17.168758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:17.170484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:17.170484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:17.170484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:17.170484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:17.170484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:17.170484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:17.170484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:17.170484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:17.170484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 311/609] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:17.170484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d414f-5f63-438a-9ef7-448dc225adb1 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:17.230295 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:17.270236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bb00f-d4c1-4c3c-a813-49781afe6b8b None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:17.305276 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d414f-5f63-438a-9ef7-448dc225adb1 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:17.331566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d2e1-39de-482d-9f6f-751a16e0567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:17.332725 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d2e1-39de-482d-9f6f-751a16e0567b None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:17.333732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d2e1-39de-482d-9f6f-751a16e0567b None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:17.333920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908d2e1-39de-482d-9f6f-751a16e0567b None admin] PATH_INFO: `/v3/roles/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:17.351868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054ab79-195b-42c4-9654-6ea70324f0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:17.352870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054ab79-195b-42c4-9654-6ea70324f0a4 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:17.353299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054ab79-195b-42c4-9654-6ea70324f0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:17.353822 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054ab79-195b-42c4-9654-6ea70324f0a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:17.356336 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908d2e1-39de-482d-9f6f-751a16e0567b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:17.358115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-1908d2e1-39de-482d-9f6f-751a16e0567b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:17.369914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:17.371045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:17.372106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:17.372106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:17.372106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:17.372106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:17.372106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:17.372106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:17.372106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:17.372106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:17.372106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:17.372106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:17.372106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:17.372106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:17.372106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 299/610] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:17.373719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d2e1-39de-482d-9f6f-751a16e0567b None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:17.375652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054ab79-195b-42c4-9654-6ea70324f0a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-3054ab79-195b-42c4-9654-6ea70324f0a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:17.381755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:17.382855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:17.383795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:17.383795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:17.383795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:17.383795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:17.383795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:17.383795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:17.383795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:17.383795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:17.383795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:17.383795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:17.383795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:17.383795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:17.387012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 312/611] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:17.388251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054ab79-195b-42c4-9654-6ea70324f0a4 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:17.529880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054ab79-195b-42c4-9654-6ea70324f0a4 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:17.557290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908d2e1-39de-482d-9f6f-751a16e0567b None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:17.575744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83c5a8-671d-4a60-a4f7-81538d31e669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:17.576803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83c5a8-671d-4a60-a4f7-81538d31e669 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:17.577362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83c5a8-671d-4a60-a4f7-81538d31e669 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:17.578060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83c5a8-671d-4a60-a4f7-81538d31e669 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:17.579241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83c5a8-671d-4a60-a4f7-81538d31e669 None admin] RBAC: Adding query filter params (name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:17.579916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83c5a8-671d-4a60-a4f7-81538d31e669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:17.582231 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83c5a8-671d-4a60-a4f7-81538d31e669 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:17.592892 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 313/612] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:17.594243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83c5a8-671d-4a60-a4f7-81538d31e669 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:17.614088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791663c-ccf8-48b5-bb60-68165d777404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:17.614759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791663c-ccf8-48b5-bb60-68165d777404 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:17.615045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791663c-ccf8-48b5-bb60-68165d777404 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:17.615322 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791663c-ccf8-48b5-bb60-68165d777404 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:17.616406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791663c-ccf8-48b5-bb60-68165d777404 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:17.616631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791663c-ccf8-48b5-bb60-68165d777404 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:17.618246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791663c-ccf8-48b5-bb60-68165d777404 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:17.621377 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:17.636529 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 300/613] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:17.637502 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791663c-ccf8-48b5-bb60-68165d777404 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:17.678232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:17.733292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83c5a8-671d-4a60-a4f7-81538d31e669 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:17.780674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafdc64-00c5-442e-8704-d749d1039df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:17.781737 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafdc64-00c5-442e-8704-d749d1039df4 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:17.782241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafdc64-00c5-442e-8704-d749d1039df4 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:17.782741 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafdc64-00c5-442e-8704-d749d1039df4 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:17.783759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791663c-ccf8-48b5-bb60-68165d777404 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:17.785202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafdc64-00c5-442e-8704-d749d1039df4 None admin] RBAC: Adding query filter params (name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:17.786330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafdc64-00c5-442e-8704-d749d1039df4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:17.788434 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafdc64-00c5-442e-8704-d749d1039df4 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:17.799787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 314/614] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:17.801031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafdc64-00c5-442e-8704-d749d1039df4 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:17.846021 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e60ca-93b1-4c3f-b865-e0388da17067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:17.848610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e60ca-93b1-4c3f-b865-e0388da17067 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:17.849347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e60ca-93b1-4c3f-b865-e0388da17067 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:17.849801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e60ca-93b1-4c3f-b865-e0388da17067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:17.876263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e60ca-93b1-4c3f-b865-e0388da17067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:17.881614 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e60ca-93b1-4c3f-b865-e0388da17067 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-d88e60ca-93b1-4c3f-b865-e0388da17067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:17.925208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:17.926420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:17.928528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:17.928528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:17.928528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:17.928528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:17.928528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 301/615] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:17.929572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e60ca-93b1-4c3f-b865-e0388da17067 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:18.008176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafdc64-00c5-442e-8704-d749d1039df4 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:18.027576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:18.053670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:18.086345 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d63753-2fd7-49f1-85ed-f1b49c9a5242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:18.092915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d63753-2fd7-49f1-85ed-f1b49c9a5242 None admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:18.093891 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d63753-2fd7-49f1-85ed-f1b49c9a5242 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:18.094668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d63753-2fd7-49f1-85ed-f1b49c9a5242 None admin] PATH_INFO: `/v3/projects/a9b328041da24cedbc801b271d6e0419/users/d39fd2708591424ca4892973e7da48ee/roles/170d72cc35f34a249c65a819a18638a4` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:18.134134 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d63753-2fd7-49f1-85ed-f1b49c9a5242 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9b328041da24cedbc801b271d6e0419, user_id=d39fd2708591424ca4892973e7da48ee, role_id=170d72cc35f34a249c65a819a18638a4)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:18.138121 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d63753-2fd7-49f1-85ed-f1b49c9a5242 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:18.144922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e60ca-93b1-4c3f-b865-e0388da17067 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:18.223391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67628c-f277-4e84-ad9f-1d0c119eb0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:18.224910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67628c-f277-4e84-ad9f-1d0c119eb0da None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:18.225235 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67628c-f277-4e84-ad9f-1d0c119eb0da None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:18.225606 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67628c-f277-4e84-ad9f-1d0c119eb0da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:18.253705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 315/616] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:41:17 2022] PUT /identity/v3/projects/a9b328041da24cedbc801b271d6e0419/users/d39fd2708591424ca4892973e7da48ee/roles/170d72cc35f34a249c65a819a18638a4 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:41:18.260972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d63753-2fd7-49f1-85ed-f1b49c9a5242 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:18.280299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67628c-f277-4e84-ad9f-1d0c119eb0da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:18.284335 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67628c-f277-4e84-ad9f-1d0c119eb0da None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-ea67628c-f277-4e84-ad9f-1d0c119eb0da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:18.325178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:18.326880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:18.328783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:18.328783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:18.328783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:18.328783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:18.328783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 302/617] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:18.334526 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67628c-f277-4e84-ad9f-1d0c119eb0da None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:18.540248 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d63753-2fd7-49f1-85ed-f1b49c9a5242 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:18.641036 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2339d-5dea-4f9a-bdff-fa4a93ca3fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:18.643187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2339d-5dea-4f9a-bdff-fa4a93ca3fef None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:18.644247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2339d-5dea-4f9a-bdff-fa4a93ca3fef None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:18.645604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2339d-5dea-4f9a-bdff-fa4a93ca3fef None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:18.647135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2339d-5dea-4f9a-bdff-fa4a93ca3fef None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:18.648038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2339d-5dea-4f9a-bdff-fa4a93ca3fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:18.651628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2339d-5dea-4f9a-bdff-fa4a93ca3fef None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:18.676415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:18.681894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67628c-f277-4e84-ad9f-1d0c119eb0da None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:18.686971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 316/618] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:18.695180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2339d-5dea-4f9a-bdff-fa4a93ca3fef None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:18.763548 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:18.777984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94094a7-96ac-4c10-a050-2f1f2e8bbd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:18.779789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94094a7-96ac-4c10-a050-2f1f2e8bbd68 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:18.780610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94094a7-96ac-4c10-a050-2f1f2e8bbd68 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:18.781371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94094a7-96ac-4c10-a050-2f1f2e8bbd68 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:18.783104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94094a7-96ac-4c10-a050-2f1f2e8bbd68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:18.785939 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94094a7-96ac-4c10-a050-2f1f2e8bbd68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:18.789046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94094a7-96ac-4c10-a050-2f1f2e8bbd68 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:18.831372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 303/619] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 498 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:18.833747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e408f-55f2-4756-9aaf-3df8acdabb29 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:18.834187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e408f-55f2-4756-9aaf-3df8acdabb29 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:18.834602 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e408f-55f2-4756-9aaf-3df8acdabb29 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:18.837435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 304/620] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:18.839201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e408f-55f2-4756-9aaf-3df8acdabb29 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:18.935843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2339d-5dea-4f9a-bdff-fa4a93ca3fef None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:19.008975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636ed21-30f0-4353-b048-c7982249fa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:19.009992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636ed21-30f0-4353-b048-c7982249fa32 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:19.010452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636ed21-30f0-4353-b048-c7982249fa32 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:19.011344 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636ed21-30f0-4353-b048-c7982249fa32 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:19.029076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636ed21-30f0-4353-b048-c7982249fa32 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:19.032085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636ed21-30f0-4353-b048-c7982249fa32 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:19.053976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e408f-55f2-4756-9aaf-3df8acdabb29 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-8636ed21-30f0-4353-b048-c7982249fa32 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:19.058359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:19.060262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:19.062093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:19.062093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:19.062093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:19.062093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:19.062093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:19.062093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:19.062093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:19.062093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:19.062093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:41:19.062093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:19.067321 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 317/621] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:41:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:19.069854 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f752e-df6a-4256-aa46-bc2766b39b86 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:19.070323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f752e-df6a-4256-aa46-bc2766b39b86 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:19.070895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f752e-df6a-4256-aa46-bc2766b39b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:19.108951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-8e2f752e-df6a-4256-aa46-bc2766b39b86 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:19.111795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-8e2f752e-df6a-4256-aa46-bc2766b39b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:19.136561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba2dfc-81e6-4e22-80d7-8242b01aa1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:19.138639 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba2dfc-81e6-4e22-80d7-8242b01aa1a2 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:19.139925 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba2dfc-81e6-4e22-80d7-8242b01aa1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:19.140695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba2dfc-81e6-4e22-80d7-8242b01aa1a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:19.159431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba2dfc-81e6-4e22-80d7-8242b01aa1a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:19.163538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba2dfc-81e6-4e22-80d7-8242b01aa1a2 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-f9ba2dfc-81e6-4e22-80d7-8242b01aa1a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:19.190774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:19.192583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:19.194435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:19.194435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:19.194435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:19.194435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:19.194435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:19.194435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:19.194435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:19.194435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:19.194435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:41:19.194435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:19.194435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 305/622] 10.222.0.94 () {56 vars in 1093 bytes} [Tue Jul 26 14:41:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:19.194435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba2dfc-81e6-4e22-80d7-8242b01aa1a2 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:19.235833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:19.279937 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 318/623] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:19.279937 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f752e-df6a-4256-aa46-bc2766b39b86 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:19.327600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:19.441200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba2dfc-81e6-4e22-80d7-8242b01aa1a2 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:19.495057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f752e-df6a-4256-aa46-bc2766b39b86 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:19.509478 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8b3e5-342a-4110-841f-34f6cb931424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:19.510858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8b3e5-342a-4110-841f-34f6cb931424 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:19.513212 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8b3e5-342a-4110-841f-34f6cb931424 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:19.514069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8b3e5-342a-4110-841f-34f6cb931424 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:19.515844 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8b3e5-342a-4110-841f-34f6cb931424 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:19.516679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8b3e5-342a-4110-841f-34f6cb931424 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:19.520422 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8b3e5-342a-4110-841f-34f6cb931424 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:19.561590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 306/624] 10.222.0.94 () {56 vars in 1135 bytes} [Tue Jul 26 14:41:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:19.563785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8b3e5-342a-4110-841f-34f6cb931424 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:19.566644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88b7bc-e910-45b9-9776-c533189da582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:19.567865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88b7bc-e910-45b9-9776-c533189da582 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:19.568413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88b7bc-e910-45b9-9776-c533189da582 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:19.568939 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88b7bc-e910-45b9-9776-c533189da582 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:19.570663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88b7bc-e910-45b9-9776-c533189da582 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:19.572542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88b7bc-e910-45b9-9776-c533189da582 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:19.574185 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88b7bc-e910-45b9-9776-c533189da582 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:19.613055 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 319/625] 10.222.0.94 () {56 vars in 1133 bytes} [Tue Jul 26 14:41:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:19.614219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88b7bc-e910-45b9-9776-c533189da582 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:19.726002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:19.769606 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8b3e5-342a-4110-841f-34f6cb931424 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:19.780813 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:19.829334 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88b7bc-e910-45b9-9776-c533189da582 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:19.838255 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8f5fa-84e5-45a3-b243-16ec2e286367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:19.839623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8f5fa-84e5-45a3-b243-16ec2e286367 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:19.840219 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8f5fa-84e5-45a3-b243-16ec2e286367 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:19.840748 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8f5fa-84e5-45a3-b243-16ec2e286367 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:19.861429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8f5fa-84e5-45a3-b243-16ec2e286367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-f5f8f5fa-84e5-45a3-b243-16ec2e286367 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:19.870831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:19.872665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:19.874284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:19.874284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:19.874284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:19.874284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:19.874284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:19.874284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:19.874284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:19.874284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:19.874284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:19.874284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:19.874284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:19.874284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:19.888136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 307/626] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:19.889461 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8f5fa-84e5-45a3-b243-16ec2e286367 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:19.910141 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a666a7-fec1-4c01-9606-5a1b843e3351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:19.912053 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a666a7-fec1-4c01-9606-5a1b843e3351 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:19.912807 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a666a7-fec1-4c01-9606-5a1b843e3351 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:19.913666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a666a7-fec1-4c01-9606-5a1b843e3351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:19.935296 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a666a7-fec1-4c01-9606-5a1b843e3351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:19.938415 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a666a7-fec1-4c01-9606-5a1b843e3351 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-56a666a7-fec1-4c01-9606-5a1b843e3351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:19.968985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:19.971173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:19.973178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:19.973178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:19.973178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:19.973178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:19.977082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 320/627] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:19.979050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a666a7-fec1-4c01-9606-5a1b843e3351 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:20.080908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8f5fa-84e5-45a3-b243-16ec2e286367 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:20.140068 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a666a7-fec1-4c01-9606-5a1b843e3351 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:20.152637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd656b-2d1a-4458-85ff-d4268e17d2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:20.153722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd656b-2d1a-4458-85ff-d4268e17d2e1 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:20.154214 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd656b-2d1a-4458-85ff-d4268e17d2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:20.154711 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd656b-2d1a-4458-85ff-d4268e17d2e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:20.169706 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:20.192928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:20.201079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97d8bf-5b25-4abb-a10f-7c673b700dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:20.202133 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97d8bf-5b25-4abb-a10f-7c673b700dc8 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:20.202658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97d8bf-5b25-4abb-a10f-7c673b700dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:20.202932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97d8bf-5b25-4abb-a10f-7c673b700dc8 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:20.203889 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97d8bf-5b25-4abb-a10f-7c673b700dc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:20.204295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97d8bf-5b25-4abb-a10f-7c673b700dc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:20.206627 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97d8bf-5b25-4abb-a10f-7c673b700dc8 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:20.222799 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 321/628] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:20.224321 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97d8bf-5b25-4abb-a10f-7c673b700dc8 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:20.304679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd656b-2d1a-4458-85ff-d4268e17d2e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:20.307176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd656b-2d1a-4458-85ff-d4268e17d2e1 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:20.420266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97d8bf-5b25-4abb-a10f-7c673b700dc8 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:20.489210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff73f64-bf2f-4ff2-8e64-7888bf71bcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:20.490029 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff73f64-bf2f-4ff2-8e64-7888bf71bcde None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:20.490484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff73f64-bf2f-4ff2-8e64-7888bf71bcde None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:20.490924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff73f64-bf2f-4ff2-8e64-7888bf71bcde None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:20.492112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff73f64-bf2f-4ff2-8e64-7888bf71bcde None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:20.492603 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff73f64-bf2f-4ff2-8e64-7888bf71bcde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:20.494970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff73f64-bf2f-4ff2-8e64-7888bf71bcde None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:20.507082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 308/629] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:41:20.507082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af506f-f7f4-4649-ae3d-44a171f8fd81 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:20.507082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af506f-f7f4-4649-ae3d-44a171f8fd81 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:20.507082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af506f-f7f4-4649-ae3d-44a171f8fd81 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:20.507865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 309/630] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:20.509449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af506f-f7f4-4649-ae3d-44a171f8fd81 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:20.524161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 322/631] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:20.524161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff73f64-bf2f-4ff2-8e64-7888bf71bcde None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:20.588598 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:20.616546 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:20.705092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af506f-f7f4-4649-ae3d-44a171f8fd81 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:20.715257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff73f64-bf2f-4ff2-8e64-7888bf71bcde None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:20.765808 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d289c5-a73c-4be8-89c9-b9743c7b7cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:20.767035 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d289c5-a73c-4be8-89c9-b9743c7b7cbe None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:20.767583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d289c5-a73c-4be8-89c9-b9743c7b7cbe None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:20.767996 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d289c5-a73c-4be8-89c9-b9743c7b7cbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:20.782507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3c114-b2d3-4691-9139-acdae81be398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:20.783698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d289c5-a73c-4be8-89c9-b9743c7b7cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:20.784387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3c114-b2d3-4691-9139-acdae81be398 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:20.784720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3c114-b2d3-4691-9139-acdae81be398 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:20.785207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3c114-b2d3-4691-9139-acdae81be398 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:20.786149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d289c5-a73c-4be8-89c9-b9743c7b7cbe None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-61d289c5-a73c-4be8-89c9-b9743c7b7cbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:20.807321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:20.809418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:20.811821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:20.811821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:20.811821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:20.811821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:20.812267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3c114-b2d3-4691-9139-acdae81be398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:20.812419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 310/632] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:20.812419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bfb94-a783-436f-bb5a-756bdf87c2b0 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:20.812991 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bfb94-a783-436f-bb5a-756bdf87c2b0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:20.813359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bfb94-a783-436f-bb5a-756bdf87c2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:20.819810 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3c114-b2d3-4691-9139-acdae81be398 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:20.838337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-6e0bfb94-a783-436f-bb5a-756bdf87c2b0 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:20.839886 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-6e0bfb94-a783-436f-bb5a-756bdf87c2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-3ee3c114-b2d3-4691-9139-acdae81be398 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:20.847683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:20.849800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:20.852653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:20.852653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:20.852653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:20.852653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:20.857654 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 323/633] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:20.859240 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3c114-b2d3-4691-9139-acdae81be398 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:20.981085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 311/634] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:20.983170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bfb94-a783-436f-bb5a-756bdf87c2b0 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:21.031184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:21.087703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3c114-b2d3-4691-9139-acdae81be398 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:21.093550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:21.161088 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb51a3-2306-4f35-b6a9-4e88e12c4af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:21.162486 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb51a3-2306-4f35-b6a9-4e88e12c4af1 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:21.163010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb51a3-2306-4f35-b6a9-4e88e12c4af1 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:21.163790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb51a3-2306-4f35-b6a9-4e88e12c4af1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:21.177840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb51a3-2306-4f35-b6a9-4e88e12c4af1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:21.180255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb51a3-2306-4f35-b6a9-4e88e12c4af1 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:21.185537 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0bfb94-a783-436f-bb5a-756bdf87c2b0 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-cedb51a3-2306-4f35-b6a9-4e88e12c4af1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:21.201684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:21.202786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:21.205169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:21.205169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:21.205169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:21.205169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:21.205169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:21.205169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:21.205169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:21.205169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:21.206507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 324/635] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:21.207309 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb51a3-2306-4f35-b6a9-4e88e12c4af1 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:21.272112 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b3c0a-6c86-4663-a419-43c649fa0a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:21.275764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b3c0a-6c86-4663-a419-43c649fa0a55 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:21.275764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b3c0a-6c86-4663-a419-43c649fa0a55 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:21.275764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b3c0a-6c86-4663-a419-43c649fa0a55 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:21.276944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b3c0a-6c86-4663-a419-43c649fa0a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:21.277521 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b3c0a-6c86-4663-a419-43c649fa0a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:21.280569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b3c0a-6c86-4663-a419-43c649fa0a55 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:21.312473 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 312/636] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:21.312473 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b3c0a-6c86-4663-a419-43c649fa0a55 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:21.436726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb51a3-2306-4f35-b6a9-4e88e12c4af1 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:21.494405 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e6eb0-4dbb-4447-be61-d0527255f43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:21.495241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e6eb0-4dbb-4447-be61-d0527255f43f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:21.495842 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e6eb0-4dbb-4447-be61-d0527255f43f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:21.496193 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e6eb0-4dbb-4447-be61-d0527255f43f None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:21.497092 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e6eb0-4dbb-4447-be61-d0527255f43f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:21.498042 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e6eb0-4dbb-4447-be61-d0527255f43f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:21.499600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e6eb0-4dbb-4447-be61-d0527255f43f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:21.525942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 325/637] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:21.525942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e6eb0-4dbb-4447-be61-d0527255f43f None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:21.552564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:21.577832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:21.586653 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b3c0a-6c86-4663-a419-43c649fa0a55 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:21.663123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c3f05-49da-4662-91a7-d8bea9fb2d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:21.664298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c3f05-49da-4662-91a7-d8bea9fb2d18 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:21.664797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c3f05-49da-4662-91a7-d8bea9fb2d18 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:21.665309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c3f05-49da-4662-91a7-d8bea9fb2d18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:21.687661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c3f05-49da-4662-91a7-d8bea9fb2d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-579c3f05-49da-4662-91a7-d8bea9fb2d18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:21.698893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:21.700948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:21.702570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:21.702570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:21.702570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:21.702570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:21.702570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:21.702570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:21.702570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:21.702570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:21.702570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:21.702570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:21.702570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:21.702570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:21.702570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 313/638] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:21.703953 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c3f05-49da-4662-91a7-d8bea9fb2d18 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:21.743090 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e6eb0-4dbb-4447-be61-d0527255f43f None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:21.808320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17406033-2463-4443-a745-b003edf57667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:21.809114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17406033-2463-4443-a745-b003edf57667 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:21.809524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17406033-2463-4443-a745-b003edf57667 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:21.809932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17406033-2463-4443-a745-b003edf57667 None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:21.811290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17406033-2463-4443-a745-b003edf57667 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:21.812055 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17406033-2463-4443-a745-b003edf57667 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:21.814785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17406033-2463-4443-a745-b003edf57667 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:21.837322 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 326/639] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:21.838617 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17406033-2463-4443-a745-b003edf57667 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:21.888209 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c3f05-49da-4662-91a7-d8bea9fb2d18 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:21.943073 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbc63e-56e4-4d5c-8a22-9e49be0661c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:21.944204 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbc63e-56e4-4d5c-8a22-9e49be0661c5 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:21.944698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbc63e-56e4-4d5c-8a22-9e49be0661c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:21.945179 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbc63e-56e4-4d5c-8a22-9e49be0661c5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:21.965252 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbc63e-56e4-4d5c-8a22-9e49be0661c5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:21.967428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbc63e-56e4-4d5c-8a22-9e49be0661c5 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:21.971686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:21.973534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-efbbc63e-56e4-4d5c-8a22-9e49be0661c5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:21.984149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:21.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:21.988170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:21.988170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:21.988170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:21.988170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:21.988170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:21.988170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:21.988170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:21.988170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:21.988170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:41:21.988170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:21.988170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 314/640] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:41:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:21.988882 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbc63e-56e4-4d5c-8a22-9e49be0661c5 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:22.006683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17406033-2463-4443-a745-b003edf57667 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:22.068114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551d5fa-e26a-46e4-912e-ab803006d87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:22.069021 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551d5fa-e26a-46e4-912e-ab803006d87b None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:22.069422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551d5fa-e26a-46e4-912e-ab803006d87b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:22.069931 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551d5fa-e26a-46e4-912e-ab803006d87b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:22.079482 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551d5fa-e26a-46e4-912e-ab803006d87b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:22.082631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551d5fa-e26a-46e4-912e-ab803006d87b None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-2551d5fa-e26a-46e4-912e-ab803006d87b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:22.105999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:22.107894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:22.118171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:22.118171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:22.118171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:22.118171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:22.118171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:22.118171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:22.118171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:22.118171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:22.118171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 327/641] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:22.118171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bcdb4-65b9-49fc-ad4c-06d812238ead None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:22.118171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bcdb4-65b9-49fc-ad4c-06d812238ead None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:22.119377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bcdb4-65b9-49fc-ad4c-06d812238ead None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:22.119377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 328/642] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:22.119377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bcdb4-65b9-49fc-ad4c-06d812238ead None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:22.162660 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbc63e-56e4-4d5c-8a22-9e49be0661c5 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:22.219732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eead91-2e94-445d-823a-65582ed0e58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:22.221719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eead91-2e94-445d-823a-65582ed0e58f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:22.222377 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eead91-2e94-445d-823a-65582ed0e58f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:22.222834 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eead91-2e94-445d-823a-65582ed0e58f None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:22.223943 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eead91-2e94-445d-823a-65582ed0e58f None admin] RBAC: Adding query filter params (name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:22.224463 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eead91-2e94-445d-823a-65582ed0e58f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:22.226983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eead91-2e94-445d-823a-65582ed0e58f None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:22.240980 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 315/643] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:22.250967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eead91-2e94-445d-823a-65582ed0e58f None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:22.303058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bcdb4-65b9-49fc-ad4c-06d812238ead None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:22.366483 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be02977-74bf-4918-8f5e-6d5ec3ffb283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:22.367602 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be02977-74bf-4918-8f5e-6d5ec3ffb283 None admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:22.368104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be02977-74bf-4918-8f5e-6d5ec3ffb283 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:22.368738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be02977-74bf-4918-8f5e-6d5ec3ffb283 None admin] PATH_INFO: `/v3/projects/a9b328041da24cedbc801b271d6e0419/users/b6cea7b5ba4a4acabe8a26837cd97310/roles/170d72cc35f34a249c65a819a18638a4` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:22.399983 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be02977-74bf-4918-8f5e-6d5ec3ffb283 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9b328041da24cedbc801b271d6e0419, user_id=b6cea7b5ba4a4acabe8a26837cd97310, role_id=170d72cc35f34a249c65a819a18638a4)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:22.402810 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be02977-74bf-4918-8f5e-6d5ec3ffb283 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:22.407322 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:22.413120 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:22.497211 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eead91-2e94-445d-823a-65582ed0e58f None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:22.514300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 329/644] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:41:22 2022] PUT /identity/v3/projects/a9b328041da24cedbc801b271d6e0419/users/b6cea7b5ba4a4acabe8a26837cd97310/roles/170d72cc35f34a249c65a819a18638a4 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:41:22.515814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be02977-74bf-4918-8f5e-6d5ec3ffb283 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:22.557871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34471b1e-eef8-4eed-b899-b55f165c7f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:22.558793 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34471b1e-eef8-4eed-b899-b55f165c7f90 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:22.559224 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34471b1e-eef8-4eed-b899-b55f165c7f90 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:22.559706 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34471b1e-eef8-4eed-b899-b55f165c7f90 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:22.561106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34471b1e-eef8-4eed-b899-b55f165c7f90 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:22.561611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34471b1e-eef8-4eed-b899-b55f165c7f90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:22.564533 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34471b1e-eef8-4eed-b899-b55f165c7f90 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:22.612910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 316/645] 10.222.0.94 () {56 vars in 1137 bytes} [Tue Jul 26 14:41:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:22.612910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c1447-8cb0-4ef3-9299-2025a5c0c430 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:22.612910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c1447-8cb0-4ef3-9299-2025a5c0c430 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:22.612910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c1447-8cb0-4ef3-9299-2025a5c0c430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:22.644815 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-206c1447-8cb0-4ef3-9299-2025a5c0c430 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:22.647064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-206c1447-8cb0-4ef3-9299-2025a5c0c430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:22.752030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be02977-74bf-4918-8f5e-6d5ec3ffb283 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:22.767718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 317/646] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:22.767718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c1447-8cb0-4ef3-9299-2025a5c0c430 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:22.832936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea70d25-5778-429a-a735-1e4b92e89922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:22.834025 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea70d25-5778-429a-a735-1e4b92e89922 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:22.834515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea70d25-5778-429a-a735-1e4b92e89922 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:22.835000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea70d25-5778-429a-a735-1e4b92e89922 None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:22.836551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea70d25-5778-429a-a735-1e4b92e89922 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:22.837116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea70d25-5778-429a-a735-1e4b92e89922 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:22.839967 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea70d25-5778-429a-a735-1e4b92e89922 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:22.865816 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 330/647] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:22.868993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:22.874042 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea70d25-5778-429a-a735-1e4b92e89922 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:22.985040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c1447-8cb0-4ef3-9299-2025a5c0c430 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:23.000607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:23.059378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5c97c-258b-4f0b-a583-07014810e5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:23.060690 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5c97c-258b-4f0b-a583-07014810e5c6 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:23.061172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5c97c-258b-4f0b-a583-07014810e5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:23.061668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5c97c-258b-4f0b-a583-07014810e5c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:23.085046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5c97c-258b-4f0b-a583-07014810e5c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:23.087995 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5c97c-258b-4f0b-a583-07014810e5c6 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:23.099563 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea70d25-5778-429a-a735-1e4b92e89922 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-e1f5c97c-258b-4f0b-a583-07014810e5c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:23.121917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:23.123779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:23.125629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:23.125629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:23.125629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:23.125629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:23.125629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 318/648] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:23.127586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5c97c-258b-4f0b-a583-07014810e5c6 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:23.177095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e499e10-9b13-4350-a39f-367e22de2595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:23.178158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e499e10-9b13-4350-a39f-367e22de2595 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:23.178631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e499e10-9b13-4350-a39f-367e22de2595 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:23.179125 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e499e10-9b13-4350-a39f-367e22de2595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:23.332367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e499e10-9b13-4350-a39f-367e22de2595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:23.335329 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e499e10-9b13-4350-a39f-367e22de2595 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:23.353007 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5c97c-258b-4f0b-a583-07014810e5c6 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:23.356131 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:23.424721 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7057bcc-63bc-403f-9a61-9f0b197050c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:23.426022 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7057bcc-63bc-403f-9a61-9f0b197050c6 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:23.426431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7057bcc-63bc-403f-9a61-9f0b197050c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:23.426843 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7057bcc-63bc-403f-9a61-9f0b197050c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:23.446524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7057bcc-63bc-403f-9a61-9f0b197050c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-c7057bcc-63bc-403f-9a61-9f0b197050c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:23.458018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:23.459281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:23.460279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:23.460279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:23.460279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:23.460279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:23.460279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:23.460279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:23.460279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:23.460279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:23.460279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:23.460279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:23.460279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:23.460279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:23.461027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 319/649] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:23.462410 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7057bcc-63bc-403f-9a61-9f0b197050c6 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:23.462767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:23.566357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 331/650] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:41:23.566357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e499e10-9b13-4350-a39f-367e22de2595 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:23.667380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7057bcc-63bc-403f-9a61-9f0b197050c6 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:23.732987 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63fb62-941f-4358-aec5-5dd5c0e3bbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:23.734015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63fb62-941f-4358-aec5-5dd5c0e3bbdb None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:23.734657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63fb62-941f-4358-aec5-5dd5c0e3bbdb None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:23.734941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63fb62-941f-4358-aec5-5dd5c0e3bbdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:23.736625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63fb62-941f-4358-aec5-5dd5c0e3bbdb None admin] RBAC: Adding query filter params (role.id=170d72cc35f34a249c65a819a18638a4, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=6a4278fe995e432d88f4346d6f54218b) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:23.737153 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63fb62-941f-4358-aec5-5dd5c0e3bbdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:23.739934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63fb62-941f-4358-aec5-5dd5c0e3bbdb None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:23.750858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e499e10-9b13-4350-a39f-367e22de2595 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:23.757322 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 320/651] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:23 2022] GET /identity/v3/role_assignments?user.id=6a4278fe995e432d88f4346d6f54218b&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=170d72cc35f34a249c65a819a18638a4 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:23.759398 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63fb62-941f-4358-aec5-5dd5c0e3bbdb None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:23.815848 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:23.820438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1ce2d-e8c8-41eb-82ce-a976d901641d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:23.821882 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1ce2d-e8c8-41eb-82ce-a976d901641d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:23.822543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1ce2d-e8c8-41eb-82ce-a976d901641d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:23.823378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1ce2d-e8c8-41eb-82ce-a976d901641d None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:23.824983 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1ce2d-e8c8-41eb-82ce-a976d901641d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:23.825577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1ce2d-e8c8-41eb-82ce-a976d901641d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:23.827804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1ce2d-e8c8-41eb-82ce-a976d901641d None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:23.847791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 332/652] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:23.849085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1ce2d-e8c8-41eb-82ce-a976d901641d None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:23.875121 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:23.979201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63fb62-941f-4358-aec5-5dd5c0e3bbdb None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:24.024459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1ce2d-e8c8-41eb-82ce-a976d901641d None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:24.044243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1760519-063d-4ec0-96b6-4430ede4a8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:24.045548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1760519-063d-4ec0-96b6-4430ede4a8ff None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:24.046052 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1760519-063d-4ec0-96b6-4430ede4a8ff None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:24.046549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1760519-063d-4ec0-96b6-4430ede4a8ff None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:24.047672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1760519-063d-4ec0-96b6-4430ede4a8ff None admin] RBAC: Adding query filter params (name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:24.048234 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1760519-063d-4ec0-96b6-4430ede4a8ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:24.051722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1760519-063d-4ec0-96b6-4430ede4a8ff None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:24.063095 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 321/653] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:24.064840 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1760519-063d-4ec0-96b6-4430ede4a8ff None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:24.100138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f14e17-98a6-469b-9b7d-2a931dfd0f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:24.101793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f14e17-98a6-469b-9b7d-2a931dfd0f30 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:24.101793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f14e17-98a6-469b-9b7d-2a931dfd0f30 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:24.101793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f14e17-98a6-469b-9b7d-2a931dfd0f30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:24.119931 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f14e17-98a6-469b-9b7d-2a931dfd0f30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:24.121970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f14e17-98a6-469b-9b7d-2a931dfd0f30 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-c9f14e17-98a6-469b-9b7d-2a931dfd0f30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:24.154798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:24.156646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:24.159009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:24.159009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:24.159009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:24.159009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:24.159009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 333/654] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:24.159009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f14e17-98a6-469b-9b7d-2a931dfd0f30 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:24.288503 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:24.309919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:24.314110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1760519-063d-4ec0-96b6-4430ede4a8ff None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:24.388956 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80d397-7fca-41cd-abf9-983333f5c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:24.390030 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80d397-7fca-41cd-abf9-983333f5c511 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:24.390475 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80d397-7fca-41cd-abf9-983333f5c511 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:24.390954 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80d397-7fca-41cd-abf9-983333f5c511 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:24.391797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f14e17-98a6-469b-9b7d-2a931dfd0f30 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:24.403388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80d397-7fca-41cd-abf9-983333f5c511 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:24.406616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80d397-7fca-41cd-abf9-983333f5c511 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-2e80d397-7fca-41cd-abf9-983333f5c511 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:24.427170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:24.428743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:24.429947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:24.429947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:24.429947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:24.429947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:24.429947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:24.429947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:24.429947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:24.429947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:24.429947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 14:41:24.429947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:24.434595 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 322/655] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:41:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:24.436230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80d397-7fca-41cd-abf9-983333f5c511 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:24.476462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16524a-0852-4fb6-8096-f619d1a52f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:24.477589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16524a-0852-4fb6-8096-f619d1a52f11 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:24.478162 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16524a-0852-4fb6-8096-f619d1a52f11 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:24.478660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16524a-0852-4fb6-8096-f619d1a52f11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:24.500845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16524a-0852-4fb6-8096-f619d1a52f11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:24.504431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16524a-0852-4fb6-8096-f619d1a52f11 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-4f16524a-0852-4fb6-8096-f619d1a52f11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:24.536367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:24.538437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:24.540446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:24.540446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:24.540446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:24.540446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:24.541623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 334/656] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:24.543538 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16524a-0852-4fb6-8096-f619d1a52f11 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:24.656904 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80d397-7fca-41cd-abf9-983333f5c511 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:24.709732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e9f17-2684-47d5-ba20-689063ffaa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:24.710757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e9f17-2684-47d5-ba20-689063ffaa39 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:24.711234 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e9f17-2684-47d5-ba20-689063ffaa39 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:24.711846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e9f17-2684-47d5-ba20-689063ffaa39 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:24.714863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e9f17-2684-47d5-ba20-689063ffaa39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:24.715650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e9f17-2684-47d5-ba20-689063ffaa39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:24.717331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e9f17-2684-47d5-ba20-689063ffaa39 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:24.746658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 323/657] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:24.746658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e9f17-2684-47d5-ba20-689063ffaa39 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:24.765107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16524a-0852-4fb6-8096-f619d1a52f11 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:24.770662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:24.796008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:24.827190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d33c98-4cda-44f8-a35f-2ea99b26b8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:24.828845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d33c98-4cda-44f8-a35f-2ea99b26b8eb None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:24.829986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d33c98-4cda-44f8-a35f-2ea99b26b8eb None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:24.830754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d33c98-4cda-44f8-a35f-2ea99b26b8eb None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:24.832379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d33c98-4cda-44f8-a35f-2ea99b26b8eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:24.832986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d33c98-4cda-44f8-a35f-2ea99b26b8eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:24.836839 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d33c98-4cda-44f8-a35f-2ea99b26b8eb None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:24.876488 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 335/658] 10.222.0.94 () {56 vars in 1137 bytes} [Tue Jul 26 14:41:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:24.877980 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d33c98-4cda-44f8-a35f-2ea99b26b8eb None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:24.956552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e9f17-2684-47d5-ba20-689063ffaa39 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:25.026974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f1354-137e-470c-b346-52bb70119d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:25.028132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f1354-137e-470c-b346-52bb70119d04 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:25.028733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f1354-137e-470c-b346-52bb70119d04 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:25.029226 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f1354-137e-470c-b346-52bb70119d04 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:25.030601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f1354-137e-470c-b346-52bb70119d04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:25.031320 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f1354-137e-470c-b346-52bb70119d04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:25.034547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f1354-137e-470c-b346-52bb70119d04 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:25.058635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 324/659] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:25.060478 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f1354-137e-470c-b346-52bb70119d04 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:25.108240 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d33c98-4cda-44f8-a35f-2ea99b26b8eb None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:25.177145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499422ec-2e2b-45a4-85f0-686101b4d66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:25.178556 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499422ec-2e2b-45a4-85f0-686101b4d66a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:25.179314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499422ec-2e2b-45a4-85f0-686101b4d66a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:25.180128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499422ec-2e2b-45a4-85f0-686101b4d66a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:25.193798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499422ec-2e2b-45a4-85f0-686101b4d66a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:25.197287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499422ec-2e2b-45a4-85f0-686101b4d66a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:25.200785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-499422ec-2e2b-45a4-85f0-686101b4d66a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:25.221258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:25.222993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:25.225269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:25.225269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:25.225269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:25.225269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:25.225269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:25.225269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:25.225269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:25.225269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:25.235931 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 336/660] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:25.238085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499422ec-2e2b-45a4-85f0-686101b4d66a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:25.278208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f1354-137e-470c-b346-52bb70119d04 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:25.313001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:25.353619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150c0d3-39fd-419d-b60a-06b04f8ad3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:25.354525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150c0d3-39fd-419d-b60a-06b04f8ad3b0 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:25.355067 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150c0d3-39fd-419d-b60a-06b04f8ad3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:25.355675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150c0d3-39fd-419d-b60a-06b04f8ad3b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:25.434521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499422ec-2e2b-45a4-85f0-686101b4d66a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:25.493616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150c0d3-39fd-419d-b60a-06b04f8ad3b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:25.495108 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150c0d3-39fd-419d-b60a-06b04f8ad3b0 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:25.510292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c63f1-70a6-49ad-8433-1bb83ade5726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:25.511969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c63f1-70a6-49ad-8433-1bb83ade5726 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:25.512913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c63f1-70a6-49ad-8433-1bb83ade5726 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:25.513735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c63f1-70a6-49ad-8433-1bb83ade5726 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:25.531018 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c63f1-70a6-49ad-8433-1bb83ade5726 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:25.534522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c63f1-70a6-49ad-8433-1bb83ade5726 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-fd2c63f1-70a6-49ad-8433-1bb83ade5726 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:25.563707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:25.566269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:25.568339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:25.568339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:25.568339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:25.568339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:25.568339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:25.568339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:25.568339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:25.568339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:25.568339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 14:41:25.568339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:25.568339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 337/661] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:25.569862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c63f1-70a6-49ad-8433-1bb83ade5726 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:25.595806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:25.660556 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 325/662] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:41:25.662731 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd651f-80e8-43da-ae72-55a88f4d6fa0 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:25.663244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd651f-80e8-43da-ae72-55a88f4d6fa0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:25.663780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd651f-80e8-43da-ae72-55a88f4d6fa0 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:25.671141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 326/663] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:25.671141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd651f-80e8-43da-ae72-55a88f4d6fa0 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:25.727012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:25.739079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c63f1-70a6-49ad-8433-1bb83ade5726 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:25.811599 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7581448-54b2-46bd-878d-26f7ac123ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:25.812716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7581448-54b2-46bd-878d-26f7ac123ca0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:25.813226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7581448-54b2-46bd-878d-26f7ac123ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:25.813726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7581448-54b2-46bd-878d-26f7ac123ca0 None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:25.815310 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7581448-54b2-46bd-878d-26f7ac123ca0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:25.815908 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7581448-54b2-46bd-878d-26f7ac123ca0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:25.818991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7581448-54b2-46bd-878d-26f7ac123ca0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:25.850980 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 338/664] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:25.850980 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7581448-54b2-46bd-878d-26f7ac123ca0 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:25.904682 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd651f-80e8-43da-ae72-55a88f4d6fa0 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:25.972850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fd62e-f401-4236-90af-4dc95a637c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:25.973624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fd62e-f401-4236-90af-4dc95a637c27 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:25.973909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fd62e-f401-4236-90af-4dc95a637c27 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:25.974189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fd62e-f401-4236-90af-4dc95a637c27 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:25.975102 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fd62e-f401-4236-90af-4dc95a637c27 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:25.975983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fd62e-f401-4236-90af-4dc95a637c27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:25.978739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fd62e-f401-4236-90af-4dc95a637c27 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:26.006212 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 327/665] 10.222.0.94 () {56 vars in 1139 bytes} [Tue Jul 26 14:41:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:26.008453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbcc13-a9ab-4359-a9dd-863d6fb71905 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:26.009386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbcc13-a9ab-4359-a9dd-863d6fb71905 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:26.009669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbcc13-a9ab-4359-a9dd-863d6fb71905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:26.034141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-febbcc13-a9ab-4359-a9dd-863d6fb71905 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:26.035858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-febbcc13-a9ab-4359-a9dd-863d6fb71905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:26.038798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7581448-54b2-46bd-878d-26f7ac123ca0 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:26.063025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:26.096551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcc2d9-1c8f-4f7c-86a2-b4795b523fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:26.097228 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcc2d9-1c8f-4f7c-86a2-b4795b523fea None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:26.097501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcc2d9-1c8f-4f7c-86a2-b4795b523fea None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:26.097776 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcc2d9-1c8f-4f7c-86a2-b4795b523fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:26.110495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bcc2d9-1c8f-4f7c-86a2-b4795b523fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:26.112358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bcc2d9-1c8f-4f7c-86a2-b4795b523fea None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-07bcc2d9-1c8f-4f7c-86a2-b4795b523fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:26.131670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:26.132749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:26.133707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:26.133707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:26.133707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:26.133707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:26.133707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 339/666] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:26.134007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcc2d9-1c8f-4f7c-86a2-b4795b523fea None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:26.146007 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 328/667] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:26.147006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbcc13-a9ab-4359-a9dd-863d6fb71905 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:26.151046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:26.334253 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcc2d9-1c8f-4f7c-86a2-b4795b523fea None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:26.357471 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbcc13-a9ab-4359-a9dd-863d6fb71905 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:26.381766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cc5f7-8968-406e-ae61-269c132549f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:26.382872 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cc5f7-8968-406e-ae61-269c132549f3 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:26.383715 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cc5f7-8968-406e-ae61-269c132549f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:26.384192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6cc5f7-8968-406e-ae61-269c132549f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:26.385222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cc5f7-8968-406e-ae61-269c132549f3 None admin] RBAC: Adding query filter params (role.id=170d72cc35f34a249c65a819a18638a4, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=7448959048f746a0ae1588ff839deb0c) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:26.385783 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cc5f7-8968-406e-ae61-269c132549f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:26.387449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6cc5f7-8968-406e-ae61-269c132549f3 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:26.397074 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 340/668] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:26 2022] GET /identity/v3/role_assignments?user.id=7448959048f746a0ae1588ff839deb0c&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=170d72cc35f34a249c65a819a18638a4 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:26.398559 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cc5f7-8968-406e-ae61-269c132549f3 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:26.424926 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d455d-8319-4917-a178-3fd469e1c7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:26.425959 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d455d-8319-4917-a178-3fd469e1c7da None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:26.426412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d455d-8319-4917-a178-3fd469e1c7da None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:26.426844 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d455d-8319-4917-a178-3fd469e1c7da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:26.473495 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:26.553094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d455d-8319-4917-a178-3fd469e1c7da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:26.555850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d455d-8319-4917-a178-3fd469e1c7da None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:26.561994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:26.617904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6cc5f7-8968-406e-ae61-269c132549f3 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:26.685618 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d290b-336d-4648-8c28-3658c32efab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:26.686573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d290b-336d-4648-8c28-3658c32efab9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:26.687009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d290b-336d-4648-8c28-3658c32efab9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:26.687448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d290b-336d-4648-8c28-3658c32efab9 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:26.688703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d290b-336d-4648-8c28-3658c32efab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:26.691952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d290b-336d-4648-8c28-3658c32efab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:26.694294 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d290b-336d-4648-8c28-3658c32efab9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:26.713587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 341/669] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:26.717556 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d290b-336d-4648-8c28-3658c32efab9 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:26.760774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 329/670] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:41:26.762316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d455d-8319-4917-a178-3fd469e1c7da None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:26.868422 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:26.887074 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d290b-336d-4648-8c28-3658c32efab9 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:26.930833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d455d-8319-4917-a178-3fd469e1c7da None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:26.952471 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:26.960464 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c44bfe-a798-4623-b79e-25e0f65cd51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:26.965502 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c44bfe-a798-4623-b79e-25e0f65cd51e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:26.965502 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c44bfe-a798-4623-b79e-25e0f65cd51e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:26.965502 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c44bfe-a798-4623-b79e-25e0f65cd51e None admin] PATH_INFO: `/v3/roles/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:26.977842 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c44bfe-a798-4623-b79e-25e0f65cd51e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:26.982904 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e9590-4fcc-46e3-85df-b2340e734198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:26.983956 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e9590-4fcc-46e3-85df-b2340e734198 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:26.984290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e9590-4fcc-46e3-85df-b2340e734198 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:26.984652 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e9590-4fcc-46e3-85df-b2340e734198 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-93c44bfe-a798-4623-b79e-25e0f65cd51e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:26.987368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:26.988520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:26.989379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:26.989379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:26.989379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:26.989379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:26.989379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:26.989379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:26.989379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:26.989379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:26.989379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:26.989379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:26.989379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:26.989379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:26.997662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e9590-4fcc-46e3-85df-b2340e734198 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:26.997662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e9590-4fcc-46e3-85df-b2340e734198 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:26.998550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 342/671] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:26.999811 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c44bfe-a798-4623-b79e-25e0f65cd51e None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-979e9590-4fcc-46e3-85df-b2340e734198 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:27.015712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:27.017799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:27.019593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:27.019593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:27.019593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:27.019593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:27.019593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:27.019593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:27.019593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:27.019593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:27.023663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 330/672] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:27.025187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e9590-4fcc-46e3-85df-b2340e734198 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:27.175320 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e9590-4fcc-46e3-85df-b2340e734198 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:27.183753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c44bfe-a798-4623-b79e-25e0f65cd51e None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:27.200031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:27.230873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170622d9-2053-40c9-bde1-116418b18233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:27.232025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170622d9-2053-40c9-bde1-116418b18233 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:27.232521 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170622d9-2053-40c9-bde1-116418b18233 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:27.233018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170622d9-2053-40c9-bde1-116418b18233 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:27.234456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170622d9-2053-40c9-bde1-116418b18233 None admin] RBAC: Adding query filter params (name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:27.235004 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170622d9-2053-40c9-bde1-116418b18233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:27.238394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170622d9-2053-40c9-bde1-116418b18233 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:27.243030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331bb03-67ff-4052-a434-6fe7ad192dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:27.244668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331bb03-67ff-4052-a434-6fe7ad192dcb None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:27.245423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331bb03-67ff-4052-a434-6fe7ad192dcb None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:27.246140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331bb03-67ff-4052-a434-6fe7ad192dcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:27.250142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 331/673] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:27.252009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170622d9-2053-40c9-bde1-116418b18233 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:27.277668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331bb03-67ff-4052-a434-6fe7ad192dcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:27.280899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331bb03-67ff-4052-a434-6fe7ad192dcb None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-b331bb03-67ff-4052-a434-6fe7ad192dcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:27.309734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:27.311000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:27.312376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:27.312376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:27.312376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:27.312376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:27.312376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 343/674] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:27.313121 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c3a43-25be-499f-b3ee-309052d2104e None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:27.313230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c3a43-25be-499f-b3ee-309052d2104e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:27.313590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c3a43-25be-499f-b3ee-309052d2104e None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:27.316311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 344/675] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:27.317192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c3a43-25be-499f-b3ee-309052d2104e None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:27.396985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:27.460877 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170622d9-2053-40c9-bde1-116418b18233 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:27.516388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8d791-34a9-4194-a5ce-7d3d43190c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:27.517688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8d791-34a9-4194-a5ce-7d3d43190c55 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:27.518191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8d791-34a9-4194-a5ce-7d3d43190c55 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:27.518695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8d791-34a9-4194-a5ce-7d3d43190c55 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:27.520229 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae8d791-34a9-4194-a5ce-7d3d43190c55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:27.520828 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae8d791-34a9-4194-a5ce-7d3d43190c55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:27.523697 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae8d791-34a9-4194-a5ce-7d3d43190c55 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:27.526548 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c3a43-25be-499f-b3ee-309052d2104e None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:27.554322 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 332/676] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:27.554322 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8d791-34a9-4194-a5ce-7d3d43190c55 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:27.598300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f948487-fb45-4018-ac9b-1b4f62475518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:27.599142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f948487-fb45-4018-ac9b-1b4f62475518 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:27.599333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f948487-fb45-4018-ac9b-1b4f62475518 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:27.599766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f948487-fb45-4018-ac9b-1b4f62475518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:27.614986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f948487-fb45-4018-ac9b-1b4f62475518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:27.615608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f948487-fb45-4018-ac9b-1b4f62475518 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:27.626852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-5f948487-fb45-4018-ac9b-1b4f62475518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:27.635653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:27.637589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:27.639326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:27.639326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:27.639326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:27.639326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:27.639326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 345/677] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:27.639326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccc12e-9227-480c-b998-b78bd0ae52d3 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:27.639977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccc12e-9227-480c-b998-b78bd0ae52d3 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:27.639977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccc12e-9227-480c-b998-b78bd0ae52d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:27.664911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-a4ccc12e-9227-480c-b998-b78bd0ae52d3 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:27.666210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a4ccc12e-9227-480c-b998-b78bd0ae52d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:27.748170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae8d791-34a9-4194-a5ce-7d3d43190c55 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:27.795507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 346/678] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:27.795507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccc12e-9227-480c-b998-b78bd0ae52d3 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:27.807265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c464b-87a6-4559-9674-9a6a0cca8372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:27.808336 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c464b-87a6-4559-9674-9a6a0cca8372 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:27.808952 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c464b-87a6-4559-9674-9a6a0cca8372 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:27.809513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c464b-87a6-4559-9674-9a6a0cca8372 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:27.810893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c464b-87a6-4559-9674-9a6a0cca8372 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:27.811426 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c464b-87a6-4559-9674-9a6a0cca8372 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:27.814415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c464b-87a6-4559-9674-9a6a0cca8372 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:27.840482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 333/679] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:27.841737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c464b-87a6-4559-9674-9a6a0cca8372 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:27.853242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:27.985113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:27.991976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c464b-87a6-4559-9674-9a6a0cca8372 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:28.003328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccc12e-9227-480c-b998-b78bd0ae52d3 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:28.058510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501a4f4-0502-449e-8543-2c6a49072d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:28.059317 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501a4f4-0502-449e-8543-2c6a49072d81 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:28.059819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501a4f4-0502-449e-8543-2c6a49072d81 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:28.060272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501a4f4-0502-449e-8543-2c6a49072d81 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:28.061171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501a4f4-0502-449e-8543-2c6a49072d81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:28.061492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501a4f4-0502-449e-8543-2c6a49072d81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:28.063810 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501a4f4-0502-449e-8543-2c6a49072d81 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:28.072842 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a52297-3513-49d1-be3b-94f4ff9d4454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:28.073629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a52297-3513-49d1-be3b-94f4ff9d4454 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:28.073952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a52297-3513-49d1-be3b-94f4ff9d4454 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:28.074267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a52297-3513-49d1-be3b-94f4ff9d4454 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:28.075273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a52297-3513-49d1-be3b-94f4ff9d4454 None admin] RBAC: Adding query filter params (role.id=170d72cc35f34a249c65a819a18638a4, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=b153935910c142aa80792e111872318a) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:28.075675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a52297-3513-49d1-be3b-94f4ff9d4454 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:28.077636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a52297-3513-49d1-be3b-94f4ff9d4454 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:28.086081 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 347/680] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:27 2022] GET /identity/v3/role_assignments?user.id=b153935910c142aa80792e111872318a&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=170d72cc35f34a249c65a819a18638a4 => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:28.087078 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a52297-3513-49d1-be3b-94f4ff9d4454 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:28.107508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 334/681] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:28.107508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501a4f4-0502-449e-8543-2c6a49072d81 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:28.295688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:28.312380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a52297-3513-49d1-be3b-94f4ff9d4454 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:28.358418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501a4f4-0502-449e-8543-2c6a49072d81 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:28.395562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df46732-3765-4644-b752-eb719c095e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:28.397366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df46732-3765-4644-b752-eb719c095e25 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:28.397860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df46732-3765-4644-b752-eb719c095e25 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:28.398358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df46732-3765-4644-b752-eb719c095e25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:28.423845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df46732-3765-4644-b752-eb719c095e25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-9df46732-3765-4644-b752-eb719c095e25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:28.434424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:28.436327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:28.438023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:28.438023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:28.438023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:28.438023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:28.438023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:28.438023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:28.438023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:28.438023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:28.438023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:28.438023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:28.438023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:41:28.438023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:28.440666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 348/682] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:41:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:28.440666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df46732-3765-4644-b752-eb719c095e25 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:28.442735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0216a99-358e-4ca3-b9af-b4733e4cdcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:28.442735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0216a99-358e-4ca3-b9af-b4733e4cdcb4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:28.443394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0216a99-358e-4ca3-b9af-b4733e4cdcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:28.443394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0216a99-358e-4ca3-b9af-b4733e4cdcb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:28.456335 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0216a99-358e-4ca3-b9af-b4733e4cdcb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:28.459271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0216a99-358e-4ca3-b9af-b4733e4cdcb4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-a0216a99-358e-4ca3-b9af-b4733e4cdcb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:28.484090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:28.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:28.488768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:28.488768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:28.488768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:28.488768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:28.488768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:28.488768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:28.488768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:28.488768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:28.488768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 335/683] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:28.496618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0216a99-358e-4ca3-b9af-b4733e4cdcb4 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:28.552016 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:28.674728 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df46732-3765-4644-b752-eb719c095e25 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:28.701393 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0216a99-358e-4ca3-b9af-b4733e4cdcb4 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:28.725018 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9670899-cbf9-42a5-819e-7673dcadd8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:28.725688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9670899-cbf9-42a5-819e-7673dcadd8dd None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:28.725970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9670899-cbf9-42a5-819e-7673dcadd8dd None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:28.726242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9670899-cbf9-42a5-819e-7673dcadd8dd None admin] PATH_INFO: `/v3/users/swift` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:28.740742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9670899-cbf9-42a5-819e-7673dcadd8dd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:28.743399 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9670899-cbf9-42a5-819e-7673dcadd8dd None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:28.750305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18caab4d-6ca5-4d07-b96d-93a59e839b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:28.752115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18caab4d-6ca5-4d07-b96d-93a59e839b74 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:28.752891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18caab4d-6ca5-4d07-b96d-93a59e839b74 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:28.753476 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18caab4d-6ca5-4d07-b96d-93a59e839b74 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:28.754538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18caab4d-6ca5-4d07-b96d-93a59e839b74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:28.755312 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18caab4d-6ca5-4d07-b96d-93a59e839b74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:28.757034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18caab4d-6ca5-4d07-b96d-93a59e839b74 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-a9670899-cbf9-42a5-819e-7673dcadd8dd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:28.760766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:28.761770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:28.762890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:28.762890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:28.762890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:28.762890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:28.762890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:28.762890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:28.762890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:28.762890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:28.762890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:41:28.762890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:28.768665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 349/684] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:41:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:28.769678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9670899-cbf9-42a5-819e-7673dcadd8dd None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:28.777984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 336/685] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:41:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:28.779492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18caab4d-6ca5-4d07-b96d-93a59e839b74 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:28.788037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:28.904715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18caab4d-6ca5-4d07-b96d-93a59e839b74 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:28.931649 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:28.942282 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9670899-cbf9-42a5-819e-7673dcadd8dd None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:28.952562 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d68492-456b-4070-b108-4835c80ab7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:28.953545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d68492-456b-4070-b108-4835c80ab7b7 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:28.953945 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d68492-456b-4070-b108-4835c80ab7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:28.954361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d68492-456b-4070-b108-4835c80ab7b7 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:28.955370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d68492-456b-4070-b108-4835c80ab7b7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:28.955831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d68492-456b-4070-b108-4835c80ab7b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:28.958187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d68492-456b-4070-b108-4835c80ab7b7 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:28.983270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 337/686] 10.222.0.94 () {56 vars in 1135 bytes} [Tue Jul 26 14:41:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:28.984490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d68492-456b-4070-b108-4835c80ab7b7 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:29.013284 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec97196-9238-4103-bdae-818459cfc45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:29.015379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec97196-9238-4103-bdae-818459cfc45a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:29.016264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec97196-9238-4103-bdae-818459cfc45a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:29.017071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec97196-9238-4103-bdae-818459cfc45a None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:29.018871 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec97196-9238-4103-bdae-818459cfc45a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:29.019891 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec97196-9238-4103-bdae-818459cfc45a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:29.023236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec97196-9238-4103-bdae-818459cfc45a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:29.057920 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 350/687] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:29.059432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec97196-9238-4103-bdae-818459cfc45a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:29.136666 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d68492-456b-4070-b108-4835c80ab7b7 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:29.186577 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014733cb-fb08-44db-b3a4-c8cc92cc2ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:29.187162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014733cb-fb08-44db-b3a4-c8cc92cc2ceb None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:29.187447 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014733cb-fb08-44db-b3a4-c8cc92cc2ceb None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:29.187789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014733cb-fb08-44db-b3a4-c8cc92cc2ceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:29.190732 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:29.198021 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:29.204872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014733cb-fb08-44db-b3a4-c8cc92cc2ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:29.206498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014733cb-fb08-44db-b3a4-c8cc92cc2ceb None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-014733cb-fb08-44db-b3a4-c8cc92cc2ceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:29.227329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:29.228524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:29.229490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:29.229490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:29.229490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:29.229490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:29.230260 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 338/688] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:29.232105 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573fa8e-06e1-483d-87b1-dd1ccc8243cc None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:29.232545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573fa8e-06e1-483d-87b1-dd1ccc8243cc None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:29.233001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573fa8e-06e1-483d-87b1-dd1ccc8243cc None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:29.235406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 339/689] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:29.237489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573fa8e-06e1-483d-87b1-dd1ccc8243cc None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:29.246875 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec97196-9238-4103-bdae-818459cfc45a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:29.308189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e37f6-0f27-40a1-b7da-6f03beceb627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:29.308890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e37f6-0f27-40a1-b7da-6f03beceb627 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:29.309206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e37f6-0f27-40a1-b7da-6f03beceb627 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:29.309483 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e37f6-0f27-40a1-b7da-6f03beceb627 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:29.398995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e37f6-0f27-40a1-b7da-6f03beceb627 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:29.400691 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e37f6-0f27-40a1-b7da-6f03beceb627 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:29.430141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573fa8e-06e1-483d-87b1-dd1ccc8243cc None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:29.498595 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a285a19-ce70-447a-93ed-e7cc3801abc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:29.499541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a285a19-ce70-447a-93ed-e7cc3801abc2 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:29.500011 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a285a19-ce70-447a-93ed-e7cc3801abc2 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:29.500492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a285a19-ce70-447a-93ed-e7cc3801abc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:29.502127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a285a19-ce70-447a-93ed-e7cc3801abc2 None admin] RBAC: Adding query filter params (role.id=170d72cc35f34a249c65a819a18638a4, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=d39fd2708591424ca4892973e7da48ee) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:29.502664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a285a19-ce70-447a-93ed-e7cc3801abc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:29.505397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a285a19-ce70-447a-93ed-e7cc3801abc2 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:29.506855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:29.522848 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 340/690] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:29 2022] GET /identity/v3/role_assignments?user.id=d39fd2708591424ca4892973e7da48ee&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=170d72cc35f34a249c65a819a18638a4 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:29.522848 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a285a19-ce70-447a-93ed-e7cc3801abc2 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:29.575955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 351/691] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:41:29.578523 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3338566-fd78-4ab0-92a4-12b25886b0fc None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:29.579094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3338566-fd78-4ab0-92a4-12b25886b0fc None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:29.579733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3338566-fd78-4ab0-92a4-12b25886b0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:29.612193 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-d3338566-fd78-4ab0-92a4-12b25886b0fc None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:29.614546 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d3338566-fd78-4ab0-92a4-12b25886b0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:29.621341 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:29.700680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a285a19-ce70-447a-93ed-e7cc3801abc2 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:29.743129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 352/692] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:29.744733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3338566-fd78-4ab0-92a4-12b25886b0fc None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:29.772003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e5022-f3b1-43ed-95f6-5b39935244fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:29.773432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e5022-f3b1-43ed-95f6-5b39935244fd None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:29.774030 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e5022-f3b1-43ed-95f6-5b39935244fd None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:29.774747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e5022-f3b1-43ed-95f6-5b39935244fd None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:29.776148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e5022-f3b1-43ed-95f6-5b39935244fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:29.776649 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e5022-f3b1-43ed-95f6-5b39935244fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:29.779094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e5022-f3b1-43ed-95f6-5b39935244fd None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:29.803126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 341/693] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:29.804604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e5022-f3b1-43ed-95f6-5b39935244fd None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:29.956129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:29.962497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3338566-fd78-4ab0-92a4-12b25886b0fc None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:29.992290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e5022-f3b1-43ed-95f6-5b39935244fd None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:30.028912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:30.039790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7cc0bb-767a-4010-9f7f-acb32fe7338c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:30.039790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cc0bb-767a-4010-9f7f-acb32fe7338c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:30.039790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cc0bb-767a-4010-9f7f-acb32fe7338c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:30.039790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cc0bb-767a-4010-9f7f-acb32fe7338c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:30.060367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7cc0bb-767a-4010-9f7f-acb32fe7338c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:30.064722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7cc0bb-767a-4010-9f7f-acb32fe7338c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:30.072356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ebb7c-1f13-4afd-a8e7-ae958a37f3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:30.073398 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ebb7c-1f13-4afd-a8e7-ae958a37f3af None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:30.073880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ebb7c-1f13-4afd-a8e7-ae958a37f3af None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:30.074346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ebb7c-1f13-4afd-a8e7-ae958a37f3af None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:30.075510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ebb7c-1f13-4afd-a8e7-ae958a37f3af None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:30.078874 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ebb7c-1f13-4afd-a8e7-ae958a37f3af None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-ba7cc0bb-767a-4010-9f7f-acb32fe7338c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:30.097932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:30.100333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:30.102462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:30.102462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:30.102462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:30.102462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:30.102462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 353/694] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-b03ebb7c-1f13-4afd-a8e7-ae958a37f3af None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:30.103109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:30.113494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:41:30.115569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:30.117423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7cc0bb-767a-4010-9f7f-acb32fe7338c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:30.117627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 342/695] 10.222.0.94 () {56 vars in 1129 bytes} [Tue Jul 26 14:41:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:30.117627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ebb7c-1f13-4afd-a8e7-ae958a37f3af None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:30.337466 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7cc0bb-767a-4010-9f7f-acb32fe7338c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:30.338287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ebb7c-1f13-4afd-a8e7-ae958a37f3af None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:30.403887 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb482db-cbfa-43d9-a8f5-ab87abe2c9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:30.404794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb482db-cbfa-43d9-a8f5-ab87abe2c9c9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:30.405102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb482db-cbfa-43d9-a8f5-ab87abe2c9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:30.405347 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb482db-cbfa-43d9-a8f5-ab87abe2c9c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:30.408842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c9bd3-5813-4247-8dcb-791d861ccef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:30.411627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9bd3-5813-4247-8dcb-791d861ccef1 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:30.412421 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9bd3-5813-4247-8dcb-791d861ccef1 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:30.413785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9bd3-5813-4247-8dcb-791d861ccef1 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:30.416118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c9bd3-5813-4247-8dcb-791d861ccef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:30.416907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c9bd3-5813-4247-8dcb-791d861ccef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:30.418522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb482db-cbfa-43d9-a8f5-ab87abe2c9c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:30.420266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb482db-cbfa-43d9-a8f5-ab87abe2c9c9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:30.420689 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c9bd3-5813-4247-8dcb-791d861ccef1 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-dbb482db-cbfa-43d9-a8f5-ab87abe2c9c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:30.445713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:30.447841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:30.449793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:30.449793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:30.449793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:30.449793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:30.449793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:30.449793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:30.449793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:30.449793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:30.449793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:41:30.449793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:30.450844 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 343/696] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:30.450844 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c9bd3-5813-4247-8dcb-791d861ccef1 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:30.454484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 354/697] 10.222.0.94 () {56 vars in 1093 bytes} [Tue Jul 26 14:41:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:30.456493 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb482db-cbfa-43d9-a8f5-ab87abe2c9c9 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:30.487366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:30.499705 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:30.578323 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c9bd3-5813-4247-8dcb-791d861ccef1 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:30.627861 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eec5cc-8d2e-4fc3-8919-c0804f80be30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:30.628799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eec5cc-8d2e-4fc3-8919-c0804f80be30 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:30.629732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eec5cc-8d2e-4fc3-8919-c0804f80be30 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:30.629911 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eec5cc-8d2e-4fc3-8919-c0804f80be30 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:30.631123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eec5cc-8d2e-4fc3-8919-c0804f80be30 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:30.631715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eec5cc-8d2e-4fc3-8919-c0804f80be30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:30.634576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eec5cc-8d2e-4fc3-8919-c0804f80be30 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:30.652077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 344/698] 10.222.0.94 () {56 vars in 1124 bytes} [Tue Jul 26 14:41:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:30.653552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eec5cc-8d2e-4fc3-8919-c0804f80be30 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:30.677260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb482db-cbfa-43d9-a8f5-ab87abe2c9c9 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:30.748832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03335a-f9f9-4c66-ab8c-83befd5e78fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:30.750818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03335a-f9f9-4c66-ab8c-83befd5e78fa None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:30.751760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03335a-f9f9-4c66-ab8c-83befd5e78fa None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:30.753333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03335a-f9f9-4c66-ab8c-83befd5e78fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:30.770009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03335a-f9f9-4c66-ab8c-83befd5e78fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:30.774279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03335a-f9f9-4c66-ab8c-83befd5e78fa None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-3c03335a-f9f9-4c66-ab8c-83befd5e78fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:30.805286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:30.807058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:30.808862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:30.808862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:30.808862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:30.808862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:30.808862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:30.808862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:30.808862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:30.808862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:30.813169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 355/699] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:30.815147 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03335a-f9f9-4c66-ab8c-83befd5e78fa None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:30.881424 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eec5cc-8d2e-4fc3-8919-c0804f80be30 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:30.884376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:30.944336 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20582022-0e22-4167-adb9-c60a3156a71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:30.945424 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20582022-0e22-4167-adb9-c60a3156a71a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:30.945726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20582022-0e22-4167-adb9-c60a3156a71a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:30.945999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20582022-0e22-4167-adb9-c60a3156a71a None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:30.946833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20582022-0e22-4167-adb9-c60a3156a71a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:30.947134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20582022-0e22-4167-adb9-c60a3156a71a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:30.948698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20582022-0e22-4167-adb9-c60a3156a71a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:30.963726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:30.984380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 345/700] 10.222.0.94 () {56 vars in 1133 bytes} [Tue Jul 26 14:41:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:30.984380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20582022-0e22-4167-adb9-c60a3156a71a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:31.011949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03335a-f9f9-4c66-ab8c-83befd5e78fa None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:31.079640 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbcef66-88e4-4c19-a208-bd17f78a3696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:31.080641 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcef66-88e4-4c19-a208-bd17f78a3696 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:31.080914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcef66-88e4-4c19-a208-bd17f78a3696 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:31.081394 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcef66-88e4-4c19-a208-bd17f78a3696 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:31.082642 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbcef66-88e4-4c19-a208-bd17f78a3696 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:31.083165 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbcef66-88e4-4c19-a208-bd17f78a3696 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:31.085766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbcef66-88e4-4c19-a208-bd17f78a3696 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:31.097925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 356/701] 10.222.0.94 () {56 vars in 1124 bytes} [Tue Jul 26 14:41:30 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:31.100324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98d681-eac0-40cd-8547-c123a0c84b39 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:31.100787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98d681-eac0-40cd-8547-c123a0c84b39 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:31.101240 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98d681-eac0-40cd-8547-c123a0c84b39 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:31.111226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 357/702] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:31 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:31.112959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98d681-eac0-40cd-8547-c123a0c84b39 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:31.187710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20582022-0e22-4167-adb9-c60a3156a71a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:31.254643 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d24b33-26a5-4fd7-ae25-aae9b861e79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:31.255763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d24b33-26a5-4fd7-ae25-aae9b861e79a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:31.256263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d24b33-26a5-4fd7-ae25-aae9b861e79a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:31.256741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d24b33-26a5-4fd7-ae25-aae9b861e79a None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:31.257751 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d24b33-26a5-4fd7-ae25-aae9b861e79a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:31.258293 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d24b33-26a5-4fd7-ae25-aae9b861e79a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:31.260284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d24b33-26a5-4fd7-ae25-aae9b861e79a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:31.278644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 346/703] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:31.280575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee956b8e-3839-4256-919f-2f01228c3278 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:31.281167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee956b8e-3839-4256-919f-2f01228c3278 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:31.281768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee956b8e-3839-4256-919f-2f01228c3278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:31.307510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ee956b8e-3839-4256-919f-2f01228c3278 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:31.310723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ee956b8e-3839-4256-919f-2f01228c3278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:31.340088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:31.367538 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98d681-eac0-40cd-8547-c123a0c84b39 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:31.441442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 347/704] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:31.442954 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee956b8e-3839-4256-919f-2f01228c3278 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:31.450861 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcc53c-ee5a-4420-bea1-1cab5f0a911b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:31.451987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcc53c-ee5a-4420-bea1-1cab5f0a911b None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:31.452481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcc53c-ee5a-4420-bea1-1cab5f0a911b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:31.452972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcc53c-ee5a-4420-bea1-1cab5f0a911b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:31.461157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:31.622287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efcc53c-ee5a-4420-bea1-1cab5f0a911b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:31.626434 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efcc53c-ee5a-4420-bea1-1cab5f0a911b None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:31.644403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee956b8e-3839-4256-919f-2f01228c3278 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:31.723026 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3f80a-1bb8-4dc9-996a-9c95f91671f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:31.724588 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3f80a-1bb8-4dc9-996a-9c95f91671f4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:31.725343 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3f80a-1bb8-4dc9-996a-9c95f91671f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:31.726072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3f80a-1bb8-4dc9-996a-9c95f91671f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:31.727728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3f80a-1bb8-4dc9-996a-9c95f91671f4 None admin] RBAC: Adding query filter params (role.id=170d72cc35f34a249c65a819a18638a4, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=b6cea7b5ba4a4acabe8a26837cd97310) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:31.728300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3f80a-1bb8-4dc9-996a-9c95f91671f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:31.731129 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3f80a-1bb8-4dc9-996a-9c95f91671f4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:31.748203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 348/705] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:31 2022] GET /identity/v3/role_assignments?user.id=b6cea7b5ba4a4acabe8a26837cd97310&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=170d72cc35f34a249c65a819a18638a4 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:31.749625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3f80a-1bb8-4dc9-996a-9c95f91671f4 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:31.816293 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 358/706] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 704 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:41:31.822217 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:31.825217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcc53c-ee5a-4420-bea1-1cab5f0a911b None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:31.949331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3f80a-1bb8-4dc9-996a-9c95f91671f4 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:31.952929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:32.000248 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cafb39-4d9e-4668-99f7-d8108d6a92fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:32.002113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cafb39-4d9e-4668-99f7-d8108d6a92fa None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:32.002969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cafb39-4d9e-4668-99f7-d8108d6a92fa None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:32.003617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cafb39-4d9e-4668-99f7-d8108d6a92fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:32.031880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cafb39-4d9e-4668-99f7-d8108d6a92fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:32.033960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cafb39-4d9e-4668-99f7-d8108d6a92fa None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-27cafb39-4d9e-4668-99f7-d8108d6a92fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:32.061225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:32.062281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:32.063162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:32.063162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:32.063162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:32.063162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:32.066323 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 349/707] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:32.075361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cafb39-4d9e-4668-99f7-d8108d6a92fa None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:32.140784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcc53c-ee5a-4420-bea1-1cab5f0a911b None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:32.198040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4e7d5-7b63-40b9-a453-682b2068e79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:32.199766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4e7d5-7b63-40b9-a453-682b2068e79e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:32.200351 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4e7d5-7b63-40b9-a453-682b2068e79e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:32.200881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4e7d5-7b63-40b9-a453-682b2068e79e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:32.218551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4e7d5-7b63-40b9-a453-682b2068e79e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:32.220910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa4e7d5-7b63-40b9-a453-682b2068e79e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:32.223355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-3aa4e7d5-7b63-40b9-a453-682b2068e79e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:32.249951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:32.250977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:32.253522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:32.253522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:32.253522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:32.253522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:32.262049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 359/708] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:32.264863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4e7d5-7b63-40b9-a453-682b2068e79e None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:32.303799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cafb39-4d9e-4668-99f7-d8108d6a92fa None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:32.352556 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40f6bc-6607-4772-882c-bd71f569ee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:32.353628 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40f6bc-6607-4772-882c-bd71f569ee7c None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:32.354183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40f6bc-6607-4772-882c-bd71f569ee7c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:32.354749 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40f6bc-6607-4772-882c-bd71f569ee7c None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:32.355842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40f6bc-6607-4772-882c-bd71f569ee7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:32.356388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40f6bc-6607-4772-882c-bd71f569ee7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:32.358574 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40f6bc-6607-4772-882c-bd71f569ee7c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:32.380258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 350/709] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:41:32.390901 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40f6bc-6607-4772-882c-bd71f569ee7c None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:32.446782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:32.482341 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4e7d5-7b63-40b9-a453-682b2068e79e None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:32.578275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e611a4e-d990-42ac-a78b-e31217369af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:32.579763 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e611a4e-d990-42ac-a78b-e31217369af0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:32.580574 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e611a4e-d990-42ac-a78b-e31217369af0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:32.581295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e611a4e-d990-42ac-a78b-e31217369af0 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:32.582775 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e611a4e-d990-42ac-a78b-e31217369af0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:32.583450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e611a4e-d990-42ac-a78b-e31217369af0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:32.586547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e611a4e-d990-42ac-a78b-e31217369af0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:32.598214 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40f6bc-6607-4772-882c-bd71f569ee7c None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:32.614583 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 360/710] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:32.617590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f88f5-14ec-4a5f-b2f8-2dda815713b0 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:32.618094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f88f5-14ec-4a5f-b2f8-2dda815713b0 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:32.618561 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f88f5-14ec-4a5f-b2f8-2dda815713b0 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:32.625256 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 361/711] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:32.625256 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f88f5-14ec-4a5f-b2f8-2dda815713b0 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:32.639512 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:32.651716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33874302-d105-4898-aa38-811195220fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:32.653335 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33874302-d105-4898-aa38-811195220fe7 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:32.653758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33874302-d105-4898-aa38-811195220fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:32.654134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33874302-d105-4898-aa38-811195220fe7 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:32.655107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33874302-d105-4898-aa38-811195220fe7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:32.656990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33874302-d105-4898-aa38-811195220fe7 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:32.754954 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 351/712] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:41:32 2022] POST /identity/v3/users => generated 254 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:41:32.759758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03085b0e-3ffa-4502-9b45-85e19a642fcf None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:32.766167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03085b0e-3ffa-4502-9b45-85e19a642fcf None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:32.766167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03085b0e-3ffa-4502-9b45-85e19a642fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:32.830695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-03085b0e-3ffa-4502-9b45-85e19a642fcf None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:32.833099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-03085b0e-3ffa-4502-9b45-85e19a642fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:32.886481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f88f5-14ec-4a5f-b2f8-2dda815713b0 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:32.968073 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fec1a-8e7f-4a59-8517-7476ae9d7797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:32.968812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fec1a-8e7f-4a59-8517-7476ae9d7797 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:32.969122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fec1a-8e7f-4a59-8517-7476ae9d7797 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:32.969397 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fec1a-8e7f-4a59-8517-7476ae9d7797 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:32.989498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fec1a-8e7f-4a59-8517-7476ae9d7797 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:32.991236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fec1a-8e7f-4a59-8517-7476ae9d7797 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:32.993222 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 352/713] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-0c6fec1a-8e7f-4a59-8517-7476ae9d7797 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:33.024221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:33.025541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:33.026667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:33.026667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:33.026667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:33.026667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:33.026667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:33.026667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:33.026667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:33.026667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:33.034269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 362/714] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:33.041284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03085b0e-3ffa-4502-9b45-85e19a642fcf None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:33.045121 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fec1a-8e7f-4a59-8517-7476ae9d7797 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:33.065643 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:33.234182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03085b0e-3ffa-4502-9b45-85e19a642fcf None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:33.277877 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fec1a-8e7f-4a59-8517-7476ae9d7797 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:33.278787 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:33.319863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465f363-c670-4004-8f44-644a59cb003c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:33.320835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465f363-c670-4004-8f44-644a59cb003c None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:33.321274 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465f363-c670-4004-8f44-644a59cb003c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:33.321708 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465f363-c670-4004-8f44-644a59cb003c None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:33.323188 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465f363-c670-4004-8f44-644a59cb003c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:33.324081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465f363-c670-4004-8f44-644a59cb003c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:33.325945 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465f363-c670-4004-8f44-644a59cb003c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:33.347580 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 353/715] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:33.351407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465f363-c670-4004-8f44-644a59cb003c None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:33.362534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae4e98-ab7a-46c4-babf-a87eac92ac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:33.363657 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae4e98-ab7a-46c4-babf-a87eac92ac0e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:33.364143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae4e98-ab7a-46c4-babf-a87eac92ac0e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:33.364631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae4e98-ab7a-46c4-babf-a87eac92ac0e None admin] PATH_INFO: `/v3/services/network` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:33.366737 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae4e98-ab7a-46c4-babf-a87eac92ac0e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:33.371536 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae4e98-ab7a-46c4-babf-a87eac92ac0e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-c4ae4e98-ab7a-46c4-babf-a87eac92ac0e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:33.395226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:33.396520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:33.397456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:33.397456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:33.397456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:33.397456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:33.397456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:33.397456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:33.397456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:33.397456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:33.397456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:33.397456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:33.397456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:41:33.397456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:33.397456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:41:33.397456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:33.398313 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 363/716] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:33 2022] GET /identity/v3/services/network => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:41:33.411449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae4e98-ab7a-46c4-babf-a87eac92ac0e None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:33.601847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:33.602120 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465f363-c670-4004-8f44-644a59cb003c None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:33.681376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae4e98-ab7a-46c4-babf-a87eac92ac0e None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:33.694861 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff06582-f530-4b9d-91b5-4d8d92575694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:33.695789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff06582-f530-4b9d-91b5-4d8d92575694 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:33.695789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff06582-f530-4b9d-91b5-4d8d92575694 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:33.696058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff06582-f530-4b9d-91b5-4d8d92575694 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:33.697042 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff06582-f530-4b9d-91b5-4d8d92575694 None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=6a4278fe995e432d88f4346d6f54218b) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:33.697346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff06582-f530-4b9d-91b5-4d8d92575694 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:33.698974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff06582-f530-4b9d-91b5-4d8d92575694 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:33.715106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 354/717] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:33 2022] GET /identity/v3/role_assignments?user.id=6a4278fe995e432d88f4346d6f54218b&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 259 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:41:33.763764 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14017366-b5e0-4581-bf6f-1cc32e10129c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:33.764477 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14017366-b5e0-4581-bf6f-1cc32e10129c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:33.764755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14017366-b5e0-4581-bf6f-1cc32e10129c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:33.765142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14017366-b5e0-4581-bf6f-1cc32e10129c None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:33.765927 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14017366-b5e0-4581-bf6f-1cc32e10129c None admin] RBAC: Adding query filter params (name=network) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:33.766293 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14017366-b5e0-4581-bf6f-1cc32e10129c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:33.769121 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14017366-b5e0-4581-bf6f-1cc32e10129c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:33.780940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 364/718] 10.222.0.94 () {56 vars in 1112 bytes} [Tue Jul 26 14:41:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:33.791584 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff06582-f530-4b9d-91b5-4d8d92575694 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:33.842320 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:33.950725 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff06582-f530-4b9d-91b5-4d8d92575694 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:34.004730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc720fa-08f6-4bfe-9730-8660a4b4b57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:34.005825 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc720fa-08f6-4bfe-9730-8660a4b4b57b None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:34.006331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc720fa-08f6-4bfe-9730-8660a4b4b57b None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:34.006822 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc720fa-08f6-4bfe-9730-8660a4b4b57b None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:34.008216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc720fa-08f6-4bfe-9730-8660a4b4b57b None admin] RBAC: Adding query filter params (type=network) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:34.008764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc720fa-08f6-4bfe-9730-8660a4b4b57b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:34.011563 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc720fa-08f6-4bfe-9730-8660a4b4b57b None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:34.027966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 355/719] 10.222.0.94 () {56 vars in 1112 bytes} [Tue Jul 26 14:41:33 2022] GET /identity/v3/services?type=network => generated 124 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:34.184301 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e49ee-09af-4421-8e93-f148581fd97a None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:34.184831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e49ee-09af-4421-8e93-f148581fd97a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:34.185257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e49ee-09af-4421-8e93-f148581fd97a None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:34.187223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 365/720] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:34.194166 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22535-4ed6-4d95-a6a3-df62aef12392 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:34.194847 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22535-4ed6-4d95-a6a3-df62aef12392 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:34.195333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22535-4ed6-4d95-a6a3-df62aef12392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:34.224684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ace22535-4ed6-4d95-a6a3-df62aef12392 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:34.227031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ace22535-4ed6-4d95-a6a3-df62aef12392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:34.335621 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 356/721] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 14:41:34.356270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e49ee-09af-4421-8e93-f148581fd97a None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:34.508919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e49ee-09af-4421-8e93-f148581fd97a None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:34.562248 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f025804-ceec-407e-b735-360b17b1a5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:34.562919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f025804-ceec-407e-b735-360b17b1a5e8 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:34.563291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f025804-ceec-407e-b735-360b17b1a5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:34.564292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f025804-ceec-407e-b735-360b17b1a5e8 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:34.565320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f025804-ceec-407e-b735-360b17b1a5e8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:34.566732 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f025804-ceec-407e-b735-360b17b1a5e8 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:34.573068 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:34.694971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 366/722] 10.222.0.94 () {60 vars in 1141 bytes} [Tue Jul 26 14:41:34 2022] POST /identity/v3/services => generated 251 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:41:35.396137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b725ca-d8fe-4381-b170-4c8c57ef4193 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:35.396967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b725ca-d8fe-4381-b170-4c8c57ef4193 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:35.397122 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b725ca-d8fe-4381-b170-4c8c57ef4193 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:35.400175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 357/723] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:35.410754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93cb7-4322-47fa-8756-75119dd89ee6 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:35.411594 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93cb7-4322-47fa-8756-75119dd89ee6 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:35.411892 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93cb7-4322-47fa-8756-75119dd89ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:35.445873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b1b93cb7-4322-47fa-8756-75119dd89ee6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:35.448115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b1b93cb7-4322-47fa-8756-75119dd89ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:35.630386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 367/724] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:35 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:35.701149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b725ca-d8fe-4381-b170-4c8c57ef4193 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:35.784804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:35.908287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b725ca-d8fe-4381-b170-4c8c57ef4193 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:35.965213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1f1fe-ce5a-4ead-bf13-075f99cc155f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:35.965863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1f1fe-ce5a-4ead-bf13-075f99cc155f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:35.966146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1f1fe-ce5a-4ead-bf13-075f99cc155f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:35.966419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1f1fe-ce5a-4ead-bf13-075f99cc155f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:35.989056 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1f1fe-ce5a-4ead-bf13-075f99cc155f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-b1b1f1fe-ce5a-4ead-bf13-075f99cc155f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:35.994342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:35.996091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:35.997672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:35.997672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:35.997672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:35.997672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:35.997672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:35.997672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:35.997672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:35.997672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:35.997672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:35.997672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:35.997672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:41:35.997672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:35.997672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 358/725] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:41:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:36.004189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b93cb7-4322-47fa-8756-75119dd89ee6 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:36.286154 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b93cb7-4322-47fa-8756-75119dd89ee6 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:36.357206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:36.385182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1f2a2-2bd3-4798-a179-64e07fb74ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:36.386078 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1f2a2-2bd3-4798-a179-64e07fb74ae3 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:36.386521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1f2a2-2bd3-4798-a179-64e07fb74ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:36.386924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1f2a2-2bd3-4798-a179-64e07fb74ae3 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:36.388035 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1f2a2-2bd3-4798-a179-64e07fb74ae3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:36.388519 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1f2a2-2bd3-4798-a179-64e07fb74ae3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:36.390901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1f2a2-2bd3-4798-a179-64e07fb74ae3 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:36.415298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 368/726] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:41:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:36.424686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1f1fe-ce5a-4ead-bf13-075f99cc155f None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:36.603181 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:36.615414 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1f1fe-ce5a-4ead-bf13-075f99cc155f None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:36.692689 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b93db-24d5-4db2-abfe-4e334efebe92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:36.694312 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b93db-24d5-4db2-abfe-4e334efebe92 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:36.695100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b93db-24d5-4db2-abfe-4e334efebe92 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:36.695938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b93db-24d5-4db2-abfe-4e334efebe92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:36.723733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b93db-24d5-4db2-abfe-4e334efebe92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:36.727053 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b93db-24d5-4db2-abfe-4e334efebe92 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-195b93db-24d5-4db2-abfe-4e334efebe92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:36.764699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:36.766998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:36.770208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:36.770208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:36.770208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:36.770208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:36.773906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 359/727] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:36.783699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1f2a2-2bd3-4798-a179-64e07fb74ae3 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:37.071017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1f2a2-2bd3-4798-a179-64e07fb74ae3 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:37.166330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb154060-8618-4e99-bb71-fa05627b6eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:37.167371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb154060-8618-4e99-bb71-fa05627b6eab None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:37.167882 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb154060-8618-4e99-bb71-fa05627b6eab None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:37.168359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb154060-8618-4e99-bb71-fa05627b6eab None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:37.169679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb154060-8618-4e99-bb71-fa05627b6eab None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:37.170214 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb154060-8618-4e99-bb71-fa05627b6eab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:37.173554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb154060-8618-4e99-bb71-fa05627b6eab None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:37.195945 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 369/728] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:37.206082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b93db-24d5-4db2-abfe-4e334efebe92 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:37.231543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912e90c-d058-4507-9fb1-cc0ea87960a2 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:37.231995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912e90c-d058-4507-9fb1-cc0ea87960a2 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:37.232464 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912e90c-d058-4507-9fb1-cc0ea87960a2 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:37.236022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 370/729] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:37.247408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583425f0-84f5-4b82-9ce9-696c747c52db None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:37.247408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583425f0-84f5-4b82-9ce9-696c747c52db None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:37.247408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583425f0-84f5-4b82-9ce9-696c747c52db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:37.295401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-583425f0-84f5-4b82-9ce9-696c747c52db None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:37.299088 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-583425f0-84f5-4b82-9ce9-696c747c52db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:37.404163 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:37.479085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 371/730] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:37.490474 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b93db-24d5-4db2-abfe-4e334efebe92 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:37.538824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701c31e-fc31-44ea-bf18-4b5d85f70973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:37.539868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701c31e-fc31-44ea-bf18-4b5d85f70973 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:37.540305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701c31e-fc31-44ea-bf18-4b5d85f70973 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:37.540777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701c31e-fc31-44ea-bf18-4b5d85f70973 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:37.551617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701c31e-fc31-44ea-bf18-4b5d85f70973 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:37.553382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583425f0-84f5-4b82-9ce9-696c747c52db None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:37.564207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701c31e-fc31-44ea-bf18-4b5d85f70973 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-0701c31e-fc31-44ea-bf18-4b5d85f70973 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:37.592564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:37.593738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:37.595630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:37.595630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:37.595630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:37.595630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:37.595630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:37.595630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:37.595630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:37.595630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:37.595630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:41:37.595630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:37.598648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 360/731] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:41:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:37.607295 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701c31e-fc31-44ea-bf18-4b5d85f70973 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:37.615684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:37.731711 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583425f0-84f5-4b82-9ce9-696c747c52db None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:37.805005 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701c31e-fc31-44ea-bf18-4b5d85f70973 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:37.823868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858e1d3-32c3-490d-b458-a1d9a85c191a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:37.824579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858e1d3-32c3-490d-b458-a1d9a85c191a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:37.824924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858e1d3-32c3-490d-b458-a1d9a85c191a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:37.825174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858e1d3-32c3-490d-b458-a1d9a85c191a None admin] PATH_INFO: `/v3/roles/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:37.855123 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858e1d3-32c3-490d-b458-a1d9a85c191a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:37.868394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef0f24-be1f-4690-b70b-d39269efba56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:37.869038 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef0f24-be1f-4690-b70b-d39269efba56 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:37.869388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef0f24-be1f-4690-b70b-d39269efba56 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:37.869891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef0f24-be1f-4690-b70b-d39269efba56 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-0858e1d3-32c3-490d-b458-a1d9a85c191a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:37.870636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:37.872473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:37.874855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:37.874855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:37.874855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:37.874855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:37.874855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:37.874855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:37.874855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:37.874855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:37.874855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:37.874855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:37.874855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:37.874855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:37.879734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef0f24-be1f-4690-b70b-d39269efba56 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:37.879734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef0f24-be1f-4690-b70b-d39269efba56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:37.879734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef0f24-be1f-4690-b70b-d39269efba56 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:37.888036 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 372/732] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:41:37.890290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90da442-16a5-47aa-8991-127cd06fd50b None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:37.890786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90da442-16a5-47aa-8991-127cd06fd50b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:37.891280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90da442-16a5-47aa-8991-127cd06fd50b None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:37.898141 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 373/733] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:37.899171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90da442-16a5-47aa-8991-127cd06fd50b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:37.915551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 361/734] 10.222.0.94 () {56 vars in 1135 bytes} [Tue Jul 26 14:41:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:41:37.926085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ded064-b6c7-446b-b36a-07bb33542cf9 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:37.926578 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ded064-b6c7-446b-b36a-07bb33542cf9 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:37.927058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ded064-b6c7-446b-b36a-07bb33542cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:37.987771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:38.112297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-04ded064-b6c7-446b-b36a-07bb33542cf9 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:38.113714 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-04ded064-b6c7-446b-b36a-07bb33542cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:38.145804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90da442-16a5-47aa-8991-127cd06fd50b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:38.202767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:38.219236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 362/735] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:38.220984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ded064-b6c7-446b-b36a-07bb33542cf9 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:38.233541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00554e9-808a-4e4a-972b-2d97de23f193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:38.235725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00554e9-808a-4e4a-972b-2d97de23f193 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:38.236572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00554e9-808a-4e4a-972b-2d97de23f193 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:38.237311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00554e9-808a-4e4a-972b-2d97de23f193 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:38.239195 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00554e9-808a-4e4a-972b-2d97de23f193 None admin] RBAC: Adding query filter params (name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:38.240042 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00554e9-808a-4e4a-972b-2d97de23f193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:38.242985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00554e9-808a-4e4a-972b-2d97de23f193 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:38.260971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 374/736] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:38.262922 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8346a28-8254-40fc-9ae5-1a370d702178 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:38.263533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8346a28-8254-40fc-9ae5-1a370d702178 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:38.264184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8346a28-8254-40fc-9ae5-1a370d702178 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:38.267163 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 375/737] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:38.272812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8346a28-8254-40fc-9ae5-1a370d702178 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:38.429972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:38.442285 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8346a28-8254-40fc-9ae5-1a370d702178 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:38.451608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ded064-b6c7-446b-b36a-07bb33542cf9 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:38.511238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1ad80-7cd7-4441-a2dd-a0e588a49c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:38.512300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1ad80-7cd7-4441-a2dd-a0e588a49c48 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:38.512772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1ad80-7cd7-4441-a2dd-a0e588a49c48 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:38.513263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1ad80-7cd7-4441-a2dd-a0e588a49c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:38.527042 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2f8cc-8734-4186-9bcb-68fe54e2ab72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:38.528154 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2f8cc-8734-4186-9bcb-68fe54e2ab72 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:38.529468 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2f8cc-8734-4186-9bcb-68fe54e2ab72 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:38.529983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2f8cc-8734-4186-9bcb-68fe54e2ab72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:38.532903 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1ad80-7cd7-4441-a2dd-a0e588a49c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:38.535981 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1ad80-7cd7-4441-a2dd-a0e588a49c48 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-04e1ad80-7cd7-4441-a2dd-a0e588a49c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:38.564043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:38.565772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:38.567457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:38.567457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:38.567457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:38.567457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:38.567902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 376/738] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:38.570218 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118fdde-4c41-47a9-b34c-e56d10aab3da None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:38.570899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118fdde-4c41-47a9-b34c-e56d10aab3da None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:38.571507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118fdde-4c41-47a9-b34c-e56d10aab3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:38.606950 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b118fdde-4c41-47a9-b34c-e56d10aab3da None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:38.608379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b118fdde-4c41-47a9-b34c-e56d10aab3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:38.643220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:38.656502 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2f8cc-8734-4186-9bcb-68fe54e2ab72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:38.659038 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2f8cc-8734-4186-9bcb-68fe54e2ab72 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:38.707509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 377/739] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 14:41:38.707509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118fdde-4c41-47a9-b34c-e56d10aab3da None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:38.817128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:38.831975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 363/740] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:38 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 14:41:38.833480 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2f8cc-8734-4186-9bcb-68fe54e2ab72 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:38.856957 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118fdde-4c41-47a9-b34c-e56d10aab3da None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:38.921938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee412c5-3f06-4f34-9ce1-b6dbfe527d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:38.923753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee412c5-3f06-4f34-9ce1-b6dbfe527d07 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:38.924230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee412c5-3f06-4f34-9ce1-b6dbfe527d07 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:38.924705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee412c5-3f06-4f34-9ce1-b6dbfe527d07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:38.938851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee412c5-3f06-4f34-9ce1-b6dbfe527d07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-9ee412c5-3f06-4f34-9ce1-b6dbfe527d07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:38.944139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:38.953089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:38.956026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:38.956026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:38.956026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:38.956026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:38.956026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:38.956026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:38.956026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:38.956026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:38.956026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:38.956026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:38.956026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:41:38.956026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:38.956026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 378/741] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:41:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:38.956026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40dc8dd-91ad-4c93-9e4c-2796e5b69320 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:38.957517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40dc8dd-91ad-4c93-9e4c-2796e5b69320 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:38.957517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40dc8dd-91ad-4c93-9e4c-2796e5b69320 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:38.957517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 379/742] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:38.957517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40dc8dd-91ad-4c93-9e4c-2796e5b69320 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:39.012587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:39.027230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2f8cc-8734-4186-9bcb-68fe54e2ab72 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:39.080495 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f8ffc-c38c-4c80-9f39-763433c758e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:39.081604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f8ffc-c38c-4c80-9f39-763433c758e1 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:39.081965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f8ffc-c38c-4c80-9f39-763433c758e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:39.082365 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f8ffc-c38c-4c80-9f39-763433c758e1 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:39.083583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f8ffc-c38c-4c80-9f39-763433c758e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:39.083998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f8ffc-c38c-4c80-9f39-763433c758e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:39.085591 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f8ffc-c38c-4c80-9f39-763433c758e1 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:39.097105 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40dc8dd-91ad-4c93-9e4c-2796e5b69320 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:39.101494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 364/743] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:39.103406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f8ffc-c38c-4c80-9f39-763433c758e1 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:39.150677 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98daec69-92f5-4626-b93f-12f960d65883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:39.151535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daec69-92f5-4626-b93f-12f960d65883 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:39.151879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daec69-92f5-4626-b93f-12f960d65883 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:39.152294 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daec69-92f5-4626-b93f-12f960d65883 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:39.153175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98daec69-92f5-4626-b93f-12f960d65883 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:39.154842 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98daec69-92f5-4626-b93f-12f960d65883 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:39.239483 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:39.251449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 380/744] 10.222.0.94 () {60 vars in 1141 bytes} [Tue Jul 26 14:41:38 2022] POST /identity/v3/services => generated 240 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:41:39.252964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98daec69-92f5-4626-b93f-12f960d65883 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:39.303950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f8ffc-c38c-4c80-9f39-763433c758e1 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:39.396792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c111eaa-df47-406a-b9f9-b89316b86dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:39.397510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c111eaa-df47-406a-b9f9-b89316b86dc7 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:39.397833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c111eaa-df47-406a-b9f9-b89316b86dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:39.398112 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c111eaa-df47-406a-b9f9-b89316b86dc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:39.413690 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c111eaa-df47-406a-b9f9-b89316b86dc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:39.415224 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c111eaa-df47-406a-b9f9-b89316b86dc7 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:39.425485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-4c111eaa-df47-406a-b9f9-b89316b86dc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:39.447921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:39.449442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:39.456710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:39.456710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:39.456710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:39.456710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:39.456710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 365/745] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:39.456710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae5d25-6077-4663-bac8-0c7e58f3deea None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:39.456710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae5d25-6077-4663-bac8-0c7e58f3deea None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:39.456710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae5d25-6077-4663-bac8-0c7e58f3deea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:39.484993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-feae5d25-6077-4663-bac8-0c7e58f3deea None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:39.488265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-feae5d25-6077-4663-bac8-0c7e58f3deea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:39.492323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98daec69-92f5-4626-b93f-12f960d65883 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:39.560583 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816b557-9f94-4310-b0ca-fe322b74d33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:39.561590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816b557-9f94-4310-b0ca-fe322b74d33c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:39.561590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816b557-9f94-4310-b0ca-fe322b74d33c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:39.561884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816b557-9f94-4310-b0ca-fe322b74d33c None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:39.562899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816b557-9f94-4310-b0ca-fe322b74d33c None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:39.564000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816b557-9f94-4310-b0ca-fe322b74d33c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:39.566628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816b557-9f94-4310-b0ca-fe322b74d33c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:39.587541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 381/746] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:41:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:39.587541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816b557-9f94-4310-b0ca-fe322b74d33c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:39.635537 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 366/747] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:41:39.635537 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae5d25-6077-4663-bac8-0c7e58f3deea None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:39.725995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:39.812207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816b557-9f94-4310-b0ca-fe322b74d33c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:39.814262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae5d25-6077-4663-bac8-0c7e58f3deea None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:39.850177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:39.861930 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a223a1e-cea2-4f46-9fca-2b8b0c19f7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:39.862693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a223a1e-cea2-4f46-9fca-2b8b0c19f7a5 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:39.862988 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a223a1e-cea2-4f46-9fca-2b8b0c19f7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:39.865172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a223a1e-cea2-4f46-9fca-2b8b0c19f7a5 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:39.865172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a223a1e-cea2-4f46-9fca-2b8b0c19f7a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:39.865172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a223a1e-cea2-4f46-9fca-2b8b0c19f7a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:39.866164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a223a1e-cea2-4f46-9fca-2b8b0c19f7a5 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:39.880416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca050d4d-169f-4cd2-ac8b-95e56b592a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:39.881816 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca050d4d-169f-4cd2-ac8b-95e56b592a2c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:39.882286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca050d4d-169f-4cd2-ac8b-95e56b592a2c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:39.882831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca050d4d-169f-4cd2-ac8b-95e56b592a2c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:39.887950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 367/748] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:39.888868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a223a1e-cea2-4f46-9fca-2b8b0c19f7a5 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:39.900496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca050d4d-169f-4cd2-ac8b-95e56b592a2c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:39.902930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca050d4d-169f-4cd2-ac8b-95e56b592a2c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-ca050d4d-169f-4cd2-ac8b-95e56b592a2c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:39.926516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:39.929058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:39.931357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:39.931357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:39.931357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:39.931357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:39.931357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:39.931357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:39.931357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:39.931357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:39.931357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:41:39.931357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:39.936675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 382/749] 10.222.0.94 () {56 vars in 1117 bytes} [Tue Jul 26 14:41:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:39.938058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca050d4d-169f-4cd2-ac8b-95e56b592a2c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:40.077965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a223a1e-cea2-4f46-9fca-2b8b0c19f7a5 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:40.107832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca050d4d-169f-4cd2-ac8b-95e56b592a2c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:40.132919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860b53c-b007-4cc7-9f5e-91749030c7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:40.133588 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860b53c-b007-4cc7-9f5e-91749030c7e4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:40.133868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860b53c-b007-4cc7-9f5e-91749030c7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:40.134145 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860b53c-b007-4cc7-9f5e-91749030c7e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:40.135173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:40.149253 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860b53c-b007-4cc7-9f5e-91749030c7e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:40.151216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860b53c-b007-4cc7-9f5e-91749030c7e4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:40.157198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65c4f4-7532-4653-90fe-b482d62a486a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:40.157965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65c4f4-7532-4653-90fe-b482d62a486a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:40.158250 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65c4f4-7532-4653-90fe-b482d62a486a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:40.158529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65c4f4-7532-4653-90fe-b482d62a486a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:40.159320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65c4f4-7532-4653-90fe-b482d62a486a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:40.161135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65c4f4-7532-4653-90fe-b482d62a486a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-3860b53c-b007-4cc7-9f5e-91749030c7e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:40.169918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:40.170951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:40.172504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:40.172504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:40.172504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:40.172504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:40.172504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 368/750] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:40.172504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860b53c-b007-4cc7-9f5e-91749030c7e4 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-0e65c4f4-7532-4653-90fe-b482d62a486a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:40.180858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:40.181873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:41:40.182717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:40.190324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 383/751] 10.222.0.94 () {56 vars in 1129 bytes} [Tue Jul 26 14:41:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:40.190324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65c4f4-7532-4653-90fe-b482d62a486a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:40.228640 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:40.343255 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860b53c-b007-4cc7-9f5e-91749030c7e4 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:40.363359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65c4f4-7532-4653-90fe-b482d62a486a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:40.400883 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbea081-5c3a-4faa-82e8-03380b2fc9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:40.401555 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbea081-5c3a-4faa-82e8-03380b2fc9e8 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:40.402004 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbea081-5c3a-4faa-82e8-03380b2fc9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:40.402330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbea081-5c3a-4faa-82e8-03380b2fc9e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:40.412183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbea081-5c3a-4faa-82e8-03380b2fc9e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:40.413819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbea081-5c3a-4faa-82e8-03380b2fc9e8 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:40.417980 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac4e55-0456-4061-bbb8-f0ca5bcc7ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:40.418621 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac4e55-0456-4061-bbb8-f0ca5bcc7ee1 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:40.418974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac4e55-0456-4061-bbb8-f0ca5bcc7ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:40.419595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac4e55-0456-4061-bbb8-f0ca5bcc7ee1 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:40.421237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac4e55-0456-4061-bbb8-f0ca5bcc7ee1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:40.421791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac4e55-0456-4061-bbb8-f0ca5bcc7ee1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:40.426104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac4e55-0456-4061-bbb8-f0ca5bcc7ee1 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-efbea081-5c3a-4faa-82e8-03380b2fc9e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:40.432981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:40.434327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:40.435999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:40.435999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:40.435999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:40.435999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:40.435999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:40.435999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:40.435999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:40.435999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:40.444100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 369/752] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:40.444100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbea081-5c3a-4faa-82e8-03380b2fc9e8 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:40.469362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 384/753] 10.222.0.94 () {56 vars in 1149 bytes} [Tue Jul 26 14:41:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:40.472138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac4e55-0456-4061-bbb8-f0ca5bcc7ee1 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:40.560669 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:40.663111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:40.669315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbea081-5c3a-4faa-82e8-03380b2fc9e8 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:40.685994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac4e55-0456-4061-bbb8-f0ca5bcc7ee1 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:40.743183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2c1c6-77b8-40c5-a0e9-b7b4cc5dedf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:40.743990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2c1c6-77b8-40c5-a0e9-b7b4cc5dedf4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:40.744273 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2c1c6-77b8-40c5-a0e9-b7b4cc5dedf4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:40.744561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2c1c6-77b8-40c5-a0e9-b7b4cc5dedf4 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:40.745385 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2c1c6-77b8-40c5-a0e9-b7b4cc5dedf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:40.745691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2c1c6-77b8-40c5-a0e9-b7b4cc5dedf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:40.745917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f575fef-7981-464e-aa46-9123527c2c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:40.746936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f575fef-7981-464e-aa46-9123527c2c7d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:40.747188 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2c1c6-77b8-40c5-a0e9-b7b4cc5dedf4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:40.747430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f575fef-7981-464e-aa46-9123527c2c7d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:40.747955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f575fef-7981-464e-aa46-9123527c2c7d None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:40.749273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f575fef-7981-464e-aa46-9123527c2c7d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:40.749808 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f575fef-7981-464e-aa46-9123527c2c7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:40.753286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f575fef-7981-464e-aa46-9123527c2c7d None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:40.771731 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 385/754] 10.222.0.94 () {56 vars in 1124 bytes} [Tue Jul 26 14:41:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:40.771731 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f575fef-7981-464e-aa46-9123527c2c7d None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:40.772135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 370/755] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:40.772135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2c1c6-77b8-40c5-a0e9-b7b4cc5dedf4 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:40.959038 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2c1c6-77b8-40c5-a0e9-b7b4cc5dedf4 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:40.966509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f575fef-7981-464e-aa46-9123527c2c7d None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:40.984063 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:41.029750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2eb65e-4577-4135-821e-77948fb65dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:41.030790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2eb65e-4577-4135-821e-77948fb65dfd None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:41.031209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2eb65e-4577-4135-821e-77948fb65dfd None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:41.031714 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2eb65e-4577-4135-821e-77948fb65dfd None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:41.031913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96000661-fddf-4236-b5b1-8cb0ff55afda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:41.033145 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96000661-fddf-4236-b5b1-8cb0ff55afda None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:41.033441 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2eb65e-4577-4135-821e-77948fb65dfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:41.033607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96000661-fddf-4236-b5b1-8cb0ff55afda None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:41.033965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96000661-fddf-4236-b5b1-8cb0ff55afda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:41.034129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2eb65e-4577-4135-821e-77948fb65dfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:41.036941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2eb65e-4577-4135-821e-77948fb65dfd None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:41.066501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 386/756] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:41.066501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2eb65e-4577-4135-821e-77948fb65dfd None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:41.083723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:41.150977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96000661-fddf-4236-b5b1-8cb0ff55afda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:41.153470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96000661-fddf-4236-b5b1-8cb0ff55afda None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:41.261209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2eb65e-4577-4135-821e-77948fb65dfd None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:41.315500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa25258-afa7-44f3-9472-d76aa4ea17dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:41.316171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa25258-afa7-44f3-9472-d76aa4ea17dd None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:41.316462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa25258-afa7-44f3-9472-d76aa4ea17dd None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:41.316779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa25258-afa7-44f3-9472-d76aa4ea17dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:41.326850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa25258-afa7-44f3-9472-d76aa4ea17dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:41.329941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa25258-afa7-44f3-9472-d76aa4ea17dd None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:41.347090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 371/757] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:41:41.347090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96000661-fddf-4236-b5b1-8cb0ff55afda None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-cfa25258-afa7-44f3-9472-d76aa4ea17dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:41.350738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:41.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:41.353560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:41.353560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:41.353560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:41.353560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:41.353560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:41.353560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:41.353560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:41.353560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:41.353560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:41:41.353560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:41.353560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 387/758] 10.222.0.94 () {56 vars in 1093 bytes} [Tue Jul 26 14:41:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:41.353560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa25258-afa7-44f3-9472-d76aa4ea17dd None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:41.393417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:41.478004 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:41.547408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa25258-afa7-44f3-9472-d76aa4ea17dd None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:41.565210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96000661-fddf-4236-b5b1-8cb0ff55afda None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:41.598372 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6486a-265d-4aaf-b01a-b7401c6541db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:41.599056 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6486a-265d-4aaf-b01a-b7401c6541db None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:41.599449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6486a-265d-4aaf-b01a-b7401c6541db None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:41.599757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6486a-265d-4aaf-b01a-b7401c6541db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:41.600590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6486a-265d-4aaf-b01a-b7401c6541db None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=b6cea7b5ba4a4acabe8a26837cd97310) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:41.600901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6486a-265d-4aaf-b01a-b7401c6541db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:41.602579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6486a-265d-4aaf-b01a-b7401c6541db None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:41.609541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 388/759] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:41 2022] GET /identity/v3/role_assignments?user.id=b6cea7b5ba4a4acabe8a26837cd97310&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:41.611545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6486a-265d-4aaf-b01a-b7401c6541db None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:41.640093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f226e6-a6b9-4b90-ac0e-7cae8982d3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:41.641235 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f226e6-a6b9-4b90-ac0e-7cae8982d3e7 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:41.641861 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f226e6-a6b9-4b90-ac0e-7cae8982d3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:41.642608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f226e6-a6b9-4b90-ac0e-7cae8982d3e7 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:41.643756 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f226e6-a6b9-4b90-ac0e-7cae8982d3e7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:41.644319 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f226e6-a6b9-4b90-ac0e-7cae8982d3e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:41.646400 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f226e6-a6b9-4b90-ac0e-7cae8982d3e7 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:41.660195 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 372/760] 10.222.0.94 () {56 vars in 1124 bytes} [Tue Jul 26 14:41:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:41.664786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f226e6-a6b9-4b90-ac0e-7cae8982d3e7 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:41.778909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:41.795653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6486a-265d-4aaf-b01a-b7401c6541db None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:41.871586 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532f799-df98-4e1f-a4fe-64126c9b4320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:41.872517 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f226e6-a6b9-4b90-ac0e-7cae8982d3e7 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:41.873632 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532f799-df98-4e1f-a4fe-64126c9b4320 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:41.874752 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532f799-df98-4e1f-a4fe-64126c9b4320 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:41.877171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532f799-df98-4e1f-a4fe-64126c9b4320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:41.910976 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532f799-df98-4e1f-a4fe-64126c9b4320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:41.912217 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:41.914605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532f799-df98-4e1f-a4fe-64126c9b4320 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:41.933626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d9b7b-499f-4194-87fc-6d50bd6e6210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:41.934620 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d9b7b-499f-4194-87fc-6d50bd6e6210 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:41.935086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d9b7b-499f-4194-87fc-6d50bd6e6210 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:41.935599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d9b7b-499f-4194-87fc-6d50bd6e6210 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:41.937276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d9b7b-499f-4194-87fc-6d50bd6e6210 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:41.937851 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d9b7b-499f-4194-87fc-6d50bd6e6210 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:41.941545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d9b7b-499f-4194-87fc-6d50bd6e6210 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-5532f799-df98-4e1f-a4fe-64126c9b4320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:41.944145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:41.946057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:41.948666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:41.948666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:41.948666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:41.948666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:41.948666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 389/761] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:41.949709 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532f799-df98-4e1f-a4fe-64126c9b4320 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:41.982086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 373/762] 10.222.0.94 () {56 vars in 1133 bytes} [Tue Jul 26 14:41:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:41.983625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d9b7b-499f-4194-87fc-6d50bd6e6210 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:42.143846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532f799-df98-4e1f-a4fe-64126c9b4320 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:42.234267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d74ac7-e39f-4ae0-a71b-074ac6075782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:42.235321 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d74ac7-e39f-4ae0-a71b-074ac6075782 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:42.235872 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d74ac7-e39f-4ae0-a71b-074ac6075782 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:42.236377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d74ac7-e39f-4ae0-a71b-074ac6075782 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:42.236844 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d9b7b-499f-4194-87fc-6d50bd6e6210 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:42.237850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d74ac7-e39f-4ae0-a71b-074ac6075782 None admin] RBAC: Adding query filter params (interface=public, service_id=2dd7073910e143e686908481a4d05f7e, region_id=RegionOne) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:42.238371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d74ac7-e39f-4ae0-a71b-074ac6075782 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:42.241021 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d74ac7-e39f-4ae0-a71b-074ac6075782 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:42.262300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 390/763] 10.222.0.94 () {56 vars in 1251 bytes} [Tue Jul 26 14:41:41 2022] GET /identity/v3/endpoints?service_id=2dd7073910e143e686908481a4d05f7e&interface=public®ion_id=RegionOne => generated 194 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:42.262300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d74ac7-e39f-4ae0-a71b-074ac6075782 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:42.288069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:42.320447 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb829d-aefc-4ba0-b17d-372c63cdf427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:42.321532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb829d-aefc-4ba0-b17d-372c63cdf427 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:42.322013 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb829d-aefc-4ba0-b17d-372c63cdf427 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:42.322693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb829d-aefc-4ba0-b17d-372c63cdf427 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:42.324134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb829d-aefc-4ba0-b17d-372c63cdf427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:42.324700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb829d-aefc-4ba0-b17d-372c63cdf427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:42.327422 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb829d-aefc-4ba0-b17d-372c63cdf427 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:42.351012 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 374/764] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:42.352543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b9b50-7968-47aa-a914-ea2d1223eac7 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:42.352927 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b9b50-7968-47aa-a914-ea2d1223eac7 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:42.353087 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b9b50-7968-47aa-a914-ea2d1223eac7 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:42.359786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 375/765] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:42.359786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b9b50-7968-47aa-a914-ea2d1223eac7 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:42.424143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:42.482069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d74ac7-e39f-4ae0-a71b-074ac6075782 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:42.522418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b9b50-7968-47aa-a914-ea2d1223eac7 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:42.555868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2919f5-b6d8-4a72-906c-c51d5b24194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:42.556576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2919f5-b6d8-4a72-906c-c51d5b24194f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:42.556931 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2919f5-b6d8-4a72-906c-c51d5b24194f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:42.557225 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2919f5-b6d8-4a72-906c-c51d5b24194f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:42.578480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2919f5-b6d8-4a72-906c-c51d5b24194f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:42.582422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2919f5-b6d8-4a72-906c-c51d5b24194f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:42.598177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619a9dc-ee66-4641-8216-94f80d1b94c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:42.599740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619a9dc-ee66-4641-8216-94f80d1b94c3 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:42.600543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619a9dc-ee66-4641-8216-94f80d1b94c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:42.601270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619a9dc-ee66-4641-8216-94f80d1b94c3 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:42.602557 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619a9dc-ee66-4641-8216-94f80d1b94c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:42.605517 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619a9dc-ee66-4641-8216-94f80d1b94c3 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-2f2919f5-b6d8-4a72-906c-c51d5b24194f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:42.608244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:42.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:42.612840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:42.612840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:42.612840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:42.612840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:42.612840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 391/766] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:42.612840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2919f5-b6d8-4a72-906c-c51d5b24194f None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:42.622986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 376/767] 10.222.0.94 () {56 vars in 1087 bytes} [Tue Jul 26 14:41:42 2022] GET /identity/v3/services => generated 772 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:42.624973 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2fb0d-f57b-478a-a67c-0198f347fb92 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:42.625552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2fb0d-f57b-478a-a67c-0198f347fb92 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:42.626735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2fb0d-f57b-478a-a67c-0198f347fb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:42.654063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-23e2fb0d-f57b-478a-a67c-0198f347fb92 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:42.658949 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-23e2fb0d-f57b-478a-a67c-0198f347fb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:42.741858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:42.769318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 377/768] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:41:42.770351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2fb0d-f57b-478a-a67c-0198f347fb92 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:42.821873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2919f5-b6d8-4a72-906c-c51d5b24194f None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:42.845436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:42.893922 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582d86c-b006-48b4-9dd5-5fb10c8984cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:42.895045 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582d86c-b006-48b4-9dd5-5fb10c8984cb None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:42.896494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582d86c-b006-48b4-9dd5-5fb10c8984cb None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:42.896494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582d86c-b006-48b4-9dd5-5fb10c8984cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:42.910534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582d86c-b006-48b4-9dd5-5fb10c8984cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:42.912742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582d86c-b006-48b4-9dd5-5fb10c8984cb None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-7582d86c-b006-48b4-9dd5-5fb10c8984cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:42.928666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:42.929730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:42.930768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:42.930768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:42.930768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:42.930768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:42.930768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:42.930768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:42.930768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:42.930768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:42.930768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 392/769] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:42.930768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582d86c-b006-48b4-9dd5-5fb10c8984cb None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:42.985558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2fb0d-f57b-478a-a67c-0198f347fb92 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:43.043374 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853bcb9-97a9-4780-a635-53b9a44b7734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:43.045085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853bcb9-97a9-4780-a635-53b9a44b7734 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:43.045085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853bcb9-97a9-4780-a635-53b9a44b7734 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:43.045085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853bcb9-97a9-4780-a635-53b9a44b7734 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:43.046188 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853bcb9-97a9-4780-a635-53b9a44b7734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:43.046600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853bcb9-97a9-4780-a635-53b9a44b7734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:43.049314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853bcb9-97a9-4780-a635-53b9a44b7734 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:43.074757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 378/770] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:43.075905 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853bcb9-97a9-4780-a635-53b9a44b7734 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:43.166009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582d86c-b006-48b4-9dd5-5fb10c8984cb None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:43.208133 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:43.255426 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f0c55-451d-4d8b-9049-c7b04f99decb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:43.256572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f0c55-451d-4d8b-9049-c7b04f99decb None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:43.257154 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f0c55-451d-4d8b-9049-c7b04f99decb None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:43.257678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f0c55-451d-4d8b-9049-c7b04f99decb None admin] PATH_INFO: `/v3/services/network` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:43.259030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f0c55-451d-4d8b-9049-c7b04f99decb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:43.260985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f0c55-451d-4d8b-9049-c7b04f99decb None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:43.275579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-236f0c55-451d-4d8b-9049-c7b04f99decb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:43.278771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:43.280072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:43.280952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:43.280952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:43.280952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:43.280952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:43.280952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:43.280952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:43.280952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:43.280952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:43.280952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:43.280952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:43.280952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:41:43.280952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:43.280952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:41:43.280952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:43.285165 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 393/771] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:42 2022] GET /identity/v3/services/network => generated 88 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:43.286915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f0c55-451d-4d8b-9049-c7b04f99decb None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:43.289277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853bcb9-97a9-4780-a635-53b9a44b7734 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:43.380033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd817b-6f6c-407d-8fff-8094c1aad734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:43.381700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd817b-6f6c-407d-8fff-8094c1aad734 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:43.382494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd817b-6f6c-407d-8fff-8094c1aad734 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:43.383113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd817b-6f6c-407d-8fff-8094c1aad734 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:43.384992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd817b-6f6c-407d-8fff-8094c1aad734 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:43.385921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd817b-6f6c-407d-8fff-8094c1aad734 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:43.388732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd817b-6f6c-407d-8fff-8094c1aad734 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:43.410634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 379/772] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:43.412971 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd817b-6f6c-407d-8fff-8094c1aad734 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:43.513883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f0c55-451d-4d8b-9049-c7b04f99decb None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:43.589118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd817b-6f6c-407d-8fff-8094c1aad734 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:43.592390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ededc-b498-42af-8fdf-81d1c2135d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:43.593911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ededc-b498-42af-8fdf-81d1c2135d67 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:43.595274 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ededc-b498-42af-8fdf-81d1c2135d67 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:43.596242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ededc-b498-42af-8fdf-81d1c2135d67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:43.606913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ededc-b498-42af-8fdf-81d1c2135d67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:43.608950 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ededc-b498-42af-8fdf-81d1c2135d67 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-9b8ededc-b498-42af-8fdf-81d1c2135d67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:43.623998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:43.625290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:43.626880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:43.626880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:43.626880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:43.626880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:43.626880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:43.626880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:43.626880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:43.626880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:43.633916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 394/773] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:43.634777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11124c4-18e8-4c3d-a581-9ce8302b7839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:43.635265 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ededc-b498-42af-8fdf-81d1c2135d67 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:43.635819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11124c4-18e8-4c3d-a581-9ce8302b7839 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:43.636178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11124c4-18e8-4c3d-a581-9ce8302b7839 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:43.636524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11124c4-18e8-4c3d-a581-9ce8302b7839 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:43.637303 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11124c4-18e8-4c3d-a581-9ce8302b7839 None admin] RBAC: Adding query filter params (name=network) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:43.637766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11124c4-18e8-4c3d-a581-9ce8302b7839 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:43.639864 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11124c4-18e8-4c3d-a581-9ce8302b7839 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:43.646093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 380/774] 10.222.0.94 () {56 vars in 1112 bytes} [Tue Jul 26 14:41:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:43.647092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11124c4-18e8-4c3d-a581-9ce8302b7839 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:43.674219 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:43.693466 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:43.772160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ededc-b498-42af-8fdf-81d1c2135d67 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:43.793176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11124c4-18e8-4c3d-a581-9ce8302b7839 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:43.822825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24667dd-74d5-4e1f-b2ad-1cce9582d876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:43.824093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24667dd-74d5-4e1f-b2ad-1cce9582d876 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:43.824571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24667dd-74d5-4e1f-b2ad-1cce9582d876 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:43.825082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24667dd-74d5-4e1f-b2ad-1cce9582d876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:43.826449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24667dd-74d5-4e1f-b2ad-1cce9582d876 None admin] RBAC: Adding query filter params (role.id=170d72cc35f34a249c65a819a18638a4, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=5b24d190536a4953930432287dfbfb9b) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:43.827050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24667dd-74d5-4e1f-b2ad-1cce9582d876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:43.829147 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24667dd-74d5-4e1f-b2ad-1cce9582d876 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:43.842793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 395/775] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:43 2022] GET /identity/v3/role_assignments?user.id=5b24d190536a4953930432287dfbfb9b&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=170d72cc35f34a249c65a819a18638a4 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:43.844063 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24667dd-74d5-4e1f-b2ad-1cce9582d876 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:43.877191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6553ac4-6b46-4b36-8de6-8dd4c030f741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:43.878650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6553ac4-6b46-4b36-8de6-8dd4c030f741 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:43.879243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6553ac4-6b46-4b36-8de6-8dd4c030f741 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:43.879873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6553ac4-6b46-4b36-8de6-8dd4c030f741 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:43.881982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6553ac4-6b46-4b36-8de6-8dd4c030f741 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:43.882644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6553ac4-6b46-4b36-8de6-8dd4c030f741 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:43.886722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6553ac4-6b46-4b36-8de6-8dd4c030f741 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:43.917776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 381/776] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:43.933026 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6553ac4-6b46-4b36-8de6-8dd4c030f741 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:44.132675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:44.184917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6553ac4-6b46-4b36-8de6-8dd4c030f741 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:44.196249 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24667dd-74d5-4e1f-b2ad-1cce9582d876 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:44.213072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:44.268659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f891a6-ffea-488b-9217-88e6b0d7bd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:44.271491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f891a6-ffea-488b-9217-88e6b0d7bd0c None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:44.271491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f891a6-ffea-488b-9217-88e6b0d7bd0c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:44.271491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f891a6-ffea-488b-9217-88e6b0d7bd0c None admin] PATH_INFO: `/v3/projects/a9b328041da24cedbc801b271d6e0419/users/6a4278fe995e432d88f4346d6f54218b/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:44.272913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f891a6-ffea-488b-9217-88e6b0d7bd0c None admin] RBAC: Authorizing `identity:create_grant(project_id=a9b328041da24cedbc801b271d6e0419, user_id=6a4278fe995e432d88f4346d6f54218b, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:44.274643 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434c323-b36a-445d-b371-944aa6ad6a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:44.274860 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f891a6-ffea-488b-9217-88e6b0d7bd0c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:44.275951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434c323-b36a-445d-b371-944aa6ad6a7b None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:44.276527 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434c323-b36a-445d-b371-944aa6ad6a7b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:44.277073 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434c323-b36a-445d-b371-944aa6ad6a7b None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:44.278416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434c323-b36a-445d-b371-944aa6ad6a7b None admin] RBAC: Adding query filter params (type=network) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:44.279053 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434c323-b36a-445d-b371-944aa6ad6a7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:44.281906 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434c323-b36a-445d-b371-944aa6ad6a7b None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:44.293891 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 396/777] 10.222.0.94 () {56 vars in 1112 bytes} [Tue Jul 26 14:41:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:44.302565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434c323-b36a-445d-b371-944aa6ad6a7b None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:44.407851 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 382/778] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:41:43 2022] PUT /identity/v3/projects/a9b328041da24cedbc801b271d6e0419/users/6a4278fe995e432d88f4346d6f54218b/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:41:44.527729 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434c323-b36a-445d-b371-944aa6ad6a7b None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:44.601425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f9427-8104-494a-b4f9-acaa20be80ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:44.602501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f9427-8104-494a-b4f9-acaa20be80ed None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:44.602994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f9427-8104-494a-b4f9-acaa20be80ed None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:44.603500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f9427-8104-494a-b4f9-acaa20be80ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:44.605008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f9427-8104-494a-b4f9-acaa20be80ed None admin] RBAC: Adding query filter params (interface=public, service_id=f403b9abd02b4a72a714f0f75827150d, region_id=RegionOne) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:44.605591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f9427-8104-494a-b4f9-acaa20be80ed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:44.608084 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f9427-8104-494a-b4f9-acaa20be80ed None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:44.620086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 397/779] 10.222.0.94 () {56 vars in 1251 bytes} [Tue Jul 26 14:41:44 2022] GET /identity/v3/endpoints?service_id=f403b9abd02b4a72a714f0f75827150d&interface=public®ion_id=RegionOne => generated 194 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:44.629494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f891a6-ffea-488b-9217-88e6b0d7bd0c None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:44.786560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f891a6-ffea-488b-9217-88e6b0d7bd0c None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:44.841032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8feef-6340-43fc-a490-face86a08658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:44.842422 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8feef-6340-43fc-a490-face86a08658 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:44.842857 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8feef-6340-43fc-a490-face86a08658 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:44.843285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8feef-6340-43fc-a490-face86a08658 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:44.844910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8feef-6340-43fc-a490-face86a08658 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:44.847115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8feef-6340-43fc-a490-face86a08658 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:44.855399 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 383/780] 10.222.0.94 () {56 vars in 1087 bytes} [Tue Jul 26 14:41:44 2022] GET /identity/v3/services => generated 772 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:45.112968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b44ac9-dc6e-4582-b785-bf8813a5d296 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:45.113196 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b44ac9-dc6e-4582-b785-bf8813a5d296 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:45.113610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b44ac9-dc6e-4582-b785-bf8813a5d296 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:45.116681 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 398/781] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:45.122684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0c6f0-58be-4b56-8b13-fde89ca9f300 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:45.123501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0c6f0-58be-4b56-8b13-fde89ca9f300 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:45.124274 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0c6f0-58be-4b56-8b13-fde89ca9f300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:45.133523 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:45.147987 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-4ab0c6f0-58be-4b56-8b13-fde89ca9f300 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:45.149581 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4ab0c6f0-58be-4b56-8b13-fde89ca9f300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:45.261012 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 384/782] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:41:45.310382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b44ac9-dc6e-4582-b785-bf8813a5d296 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:45.393028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:45.534426 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b44ac9-dc6e-4582-b785-bf8813a5d296 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:45.616743 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30183f-b893-44fb-be51-ddba913de11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:45.618109 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30183f-b893-44fb-be51-ddba913de11f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:45.618591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30183f-b893-44fb-be51-ddba913de11f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:45.619077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30183f-b893-44fb-be51-ddba913de11f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:45.638575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842385da-111b-46b6-b121-775ffc2ada9a None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:45.639176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842385da-111b-46b6-b121-775ffc2ada9a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:45.639736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842385da-111b-46b6-b121-775ffc2ada9a None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:45.642020 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 385/783] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:45.647584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30183f-b893-44fb-be51-ddba913de11f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:45.653162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145306ca-f1cc-4084-bbc3-c8ba70692004 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:45.653447 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145306ca-f1cc-4084-bbc3-c8ba70692004 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:45.654000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145306ca-f1cc-4084-bbc3-c8ba70692004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-fb30183f-b893-44fb-be51-ddba913de11f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:45.654517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:45.655455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:45.656353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:45.656353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:45.656353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:45.656353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:45.656353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:45.656353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:45.656353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:45.656353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:45.656353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:45.656353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:45.656353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:41:45.656353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:45.656353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 399/784] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:41:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:45.664456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30183f-b893-44fb-be51-ddba913de11f None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:45.711793 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-145306ca-f1cc-4084-bbc3-c8ba70692004 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:45.717575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-145306ca-f1cc-4084-bbc3-c8ba70692004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:45.855173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:45.870406 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30183f-b893-44fb-be51-ddba913de11f None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:45.893851 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 386/785] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:41:45.929610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145306ca-f1cc-4084-bbc3-c8ba70692004 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:45.948635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f4f43-877e-4d04-9c14-07d52d2e23d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:45.949307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f4f43-877e-4d04-9c14-07d52d2e23d8 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:45.949585 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f4f43-877e-4d04-9c14-07d52d2e23d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:45.949895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f4f43-877e-4d04-9c14-07d52d2e23d8 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:45.950667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f4f43-877e-4d04-9c14-07d52d2e23d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:45.950970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f4f43-877e-4d04-9c14-07d52d2e23d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:45.952540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f4f43-877e-4d04-9c14-07d52d2e23d8 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:45.975517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 400/786] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:41:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:45.985250 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f4f43-877e-4d04-9c14-07d52d2e23d8 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:46.040781 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:46.127285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145306ca-f1cc-4084-bbc3-c8ba70692004 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:46.229265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb78441-fe75-454b-8969-1df57c358dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:46.230120 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb78441-fe75-454b-8969-1df57c358dd4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:46.230409 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb78441-fe75-454b-8969-1df57c358dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:46.230912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb78441-fe75-454b-8969-1df57c358dd4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:46.232965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb78441-fe75-454b-8969-1df57c358dd4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:46.236340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb78441-fe75-454b-8969-1df57c358dd4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:46.258827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f4f43-877e-4d04-9c14-07d52d2e23d8 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-bbb78441-fe75-454b-8969-1df57c358dd4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:46.283186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:46.289469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:46.290616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 387/787] 10.222.0.94 () {56 vars in 1129 bytes} [Tue Jul 26 14:41:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:41:46.293236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb78441-fe75-454b-8969-1df57c358dd4 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:46.316566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8255e9-ecba-45ee-86d7-347b214388d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:46.323085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8255e9-ecba-45ee-86d7-347b214388d8 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:46.323085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8255e9-ecba-45ee-86d7-347b214388d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:46.323085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8255e9-ecba-45ee-86d7-347b214388d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:46.345996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8255e9-ecba-45ee-86d7-347b214388d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:46.351694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8255e9-ecba-45ee-86d7-347b214388d8 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-0e8255e9-ecba-45ee-86d7-347b214388d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:46.400769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:46.402564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:46.405520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:46.405520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:46.405520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:46.405520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:46.410839 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 401/788] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:41:46.421962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8255e9-ecba-45ee-86d7-347b214388d8 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:46.445391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:46.621839 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb78441-fe75-454b-8969-1df57c358dd4 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:46.650297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8255e9-ecba-45ee-86d7-347b214388d8 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:46.679668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:46.712201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63243a2b-f2a2-49f7-b1c0-557697d94cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:46.714190 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243a2b-f2a2-49f7-b1c0-557697d94cab None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:46.717128 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243a2b-f2a2-49f7-b1c0-557697d94cab None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:46.723600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243a2b-f2a2-49f7-b1c0-557697d94cab None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:46.725378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63243a2b-f2a2-49f7-b1c0-557697d94cab None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:46.726248 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63243a2b-f2a2-49f7-b1c0-557697d94cab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:46.729545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63243a2b-f2a2-49f7-b1c0-557697d94cab None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:46.746685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 388/789] 10.222.0.94 () {56 vars in 1124 bytes} [Tue Jul 26 14:41:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:46.755776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63243a2b-f2a2-49f7-b1c0-557697d94cab None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:46.794434 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5be2ac-29e7-47b5-b209-6f5cbd9b1517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:46.794926 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5be2ac-29e7-47b5-b209-6f5cbd9b1517 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:46.795421 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5be2ac-29e7-47b5-b209-6f5cbd9b1517 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:46.795954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5be2ac-29e7-47b5-b209-6f5cbd9b1517 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:46.797363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5be2ac-29e7-47b5-b209-6f5cbd9b1517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:46.797909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5be2ac-29e7-47b5-b209-6f5cbd9b1517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:46.800638 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5be2ac-29e7-47b5-b209-6f5cbd9b1517 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:46.825809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 402/790] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:41:46.837189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5be2ac-29e7-47b5-b209-6f5cbd9b1517 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:46.947049 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63243a2b-f2a2-49f7-b1c0-557697d94cab None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:47.031239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286348a-71e2-47da-a868-5fe0497f5079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:47.032350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286348a-71e2-47da-a868-5fe0497f5079 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:47.032350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286348a-71e2-47da-a868-5fe0497f5079 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:47.032712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286348a-71e2-47da-a868-5fe0497f5079 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:47.033638 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286348a-71e2-47da-a868-5fe0497f5079 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:47.034194 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286348a-71e2-47da-a868-5fe0497f5079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:47.044264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286348a-71e2-47da-a868-5fe0497f5079 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:47.050456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:47.062511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 389/791] 10.222.0.94 () {56 vars in 1124 bytes} [Tue Jul 26 14:41:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:47.069725 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286348a-71e2-47da-a868-5fe0497f5079 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:47.123285 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5be2ac-29e7-47b5-b209-6f5cbd9b1517 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:47.216332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7fafe-2054-470e-bff0-79a89209127a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:47.217011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7fafe-2054-470e-bff0-79a89209127a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:47.217510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7fafe-2054-470e-bff0-79a89209127a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:47.217986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7fafe-2054-470e-bff0-79a89209127a None admin] PATH_INFO: `/v3/users/swift` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:47.234058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7fafe-2054-470e-bff0-79a89209127a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:47.236299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7fafe-2054-470e-bff0-79a89209127a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:47.264050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-fdf7fafe-2054-470e-bff0-79a89209127a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:47.283885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:47.285605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:47.286722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:47.286722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:47.286722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:47.286722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:47.286722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:47.286722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:47.286722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:47.286722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:47.286722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:41:47.286722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:47.287409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 403/792] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:41:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:47.299431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286348a-71e2-47da-a868-5fe0497f5079 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:47.301206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7fafe-2054-470e-bff0-79a89209127a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:47.378015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaed89d-666a-4772-92aa-48a389dab4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:47.378917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaed89d-666a-4772-92aa-48a389dab4ae None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:47.379251 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaed89d-666a-4772-92aa-48a389dab4ae None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:47.379731 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaed89d-666a-4772-92aa-48a389dab4ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:47.380452 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaed89d-666a-4772-92aa-48a389dab4ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:47.381996 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaed89d-666a-4772-92aa-48a389dab4ae None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:47.539684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 390/793] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 14:41:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:41:47.554934 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7fafe-2054-470e-bff0-79a89209127a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:47.615112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:47.641109 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0a5e7-ebdc-47b9-b2c5-c1c22ef699a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:47.641846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0a5e7-ebdc-47b9-b2c5-c1c22ef699a9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:47.642289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0a5e7-ebdc-47b9-b2c5-c1c22ef699a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:47.642605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0a5e7-ebdc-47b9-b2c5-c1c22ef699a9 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:47.643624 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0a5e7-ebdc-47b9-b2c5-c1c22ef699a9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:47.644129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0a5e7-ebdc-47b9-b2c5-c1c22ef699a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:47.646021 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0a5e7-ebdc-47b9-b2c5-c1c22ef699a9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:47.677323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 404/794] 10.222.0.94 () {56 vars in 1135 bytes} [Tue Jul 26 14:41:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:47.684170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaed89d-666a-4772-92aa-48a389dab4ae None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:47.756389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb5609-5fb2-4e29-badc-0e00f62f8ad6 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:47.756858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb5609-5fb2-4e29-badc-0e00f62f8ad6 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:47.757114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb5609-5fb2-4e29-badc-0e00f62f8ad6 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:47.758964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 405/795] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:41:47.770320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda5b4f-2276-45ab-8da7-eea61258741d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:47.770585 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda5b4f-2276-45ab-8da7-eea61258741d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:47.771027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda5b4f-2276-45ab-8da7-eea61258741d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:47.821409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7eda5b4f-2276-45ab-8da7-eea61258741d None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:47.822795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7eda5b4f-2276-45ab-8da7-eea61258741d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:47.946922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaed89d-666a-4772-92aa-48a389dab4ae None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:47.975365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 406/796] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:41:47.978738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:47.982112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f89a5a-40a3-4374-aecb-bf63b8f1675b None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:47.982602 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f89a5a-40a3-4374-aecb-bf63b8f1675b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:47.983091 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f89a5a-40a3-4374-aecb-bf63b8f1675b None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:47.986491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 407/797] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:47.992573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece0b1d-0595-461d-aaa9-7654b6dcfb39 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:47.993090 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece0b1d-0595-461d-aaa9-7654b6dcfb39 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:47.993269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece0b1d-0595-461d-aaa9-7654b6dcfb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:48.027330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446014a-fbec-46ab-bbb4-7fbbd504cdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:48.028070 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446014a-fbec-46ab-bbb4-7fbbd504cdd5 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:48.028343 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446014a-fbec-46ab-bbb4-7fbbd504cdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:48.028625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446014a-fbec-46ab-bbb4-7fbbd504cdd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:48.032823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-1ece0b1d-0595-461d-aaa9-7654b6dcfb39 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:48.035497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1ece0b1d-0595-461d-aaa9-7654b6dcfb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:48.048870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446014a-fbec-46ab-bbb4-7fbbd504cdd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:48.051710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446014a-fbec-46ab-bbb4-7fbbd504cdd5 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-5446014a-fbec-46ab-bbb4-7fbbd504cdd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:48.088059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:48.089798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:48.092611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:48.092611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:48.092611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:48.092611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:48.092611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 391/798] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:48.092611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446014a-fbec-46ab-bbb4-7fbbd504cdd5 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:48.197593 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 408/799] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:41:48.199771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece0b1d-0595-461d-aaa9-7654b6dcfb39 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:48.301493 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:48.357719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446014a-fbec-46ab-bbb4-7fbbd504cdd5 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:48.402706 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f620e-dd2a-4ed3-9a74-43a0f88c8279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:48.403601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f620e-dd2a-4ed3-9a74-43a0f88c8279 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:48.404013 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f620e-dd2a-4ed3-9a74-43a0f88c8279 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:48.404359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f620e-dd2a-4ed3-9a74-43a0f88c8279 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:48.416106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece0b1d-0595-461d-aaa9-7654b6dcfb39 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:48.419021 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f620e-dd2a-4ed3-9a74-43a0f88c8279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-656f620e-dd2a-4ed3-9a74-43a0f88c8279 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:48.426304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:48.428428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:48.430356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:48.430356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:48.430356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:48.430356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:48.430356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:48.430356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:48.430356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:48.430356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:48.430356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:48.430356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:48.430356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:48.430356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:48.433326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 392/800] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:48.435121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f620e-dd2a-4ed3-9a74-43a0f88c8279 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:48.486631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:48.505353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1bb13-1f2d-4f42-b113-f623d6a0b465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:48.506699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1bb13-1f2d-4f42-b113-f623d6a0b465 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:48.507918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1bb13-1f2d-4f42-b113-f623d6a0b465 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:48.512596 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1bb13-1f2d-4f42-b113-f623d6a0b465 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:48.513864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1bb13-1f2d-4f42-b113-f623d6a0b465 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:48.514275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1bb13-1f2d-4f42-b113-f623d6a0b465 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:48.515797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1bb13-1f2d-4f42-b113-f623d6a0b465 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:48.530004 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 409/801] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:48.532248 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1bb13-1f2d-4f42-b113-f623d6a0b465 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:48.620739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f620e-dd2a-4ed3-9a74-43a0f88c8279 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:48.665282 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1bb13-1f2d-4f42-b113-f623d6a0b465 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:48.667305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:48.673722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed159e12-a279-4a0d-b3ce-01aab4f8ff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:48.674584 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed159e12-a279-4a0d-b3ce-01aab4f8ff09 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:48.675176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed159e12-a279-4a0d-b3ce-01aab4f8ff09 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:48.675610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed159e12-a279-4a0d-b3ce-01aab4f8ff09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:48.690757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed159e12-a279-4a0d-b3ce-01aab4f8ff09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-ed159e12-a279-4a0d-b3ce-01aab4f8ff09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:48.698198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:48.700152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:48.702477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:48.702477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:48.702477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:48.702477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:48.702477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:48.702477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:48.702477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:48.702477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:48.702477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:48.702477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:48.702477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:41:48.702477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:48.702477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 393/802] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:41:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:48.702477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975970b-819b-4e53-9263-211fe710459b None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:48.705863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975970b-819b-4e53-9263-211fe710459b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:48.705863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975970b-819b-4e53-9263-211fe710459b None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:48.711504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 394/803] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:48.711504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975970b-819b-4e53-9263-211fe710459b None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:48.724884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf55a09-5a70-4d78-88ba-60712046ad6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:48.725919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf55a09-5a70-4d78-88ba-60712046ad6e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:48.726335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf55a09-5a70-4d78-88ba-60712046ad6e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:48.726796 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf55a09-5a70-4d78-88ba-60712046ad6e None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:48.727794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf55a09-5a70-4d78-88ba-60712046ad6e None admin] RBAC: Adding query filter params (name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:48.728271 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf55a09-5a70-4d78-88ba-60712046ad6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:48.730252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf55a09-5a70-4d78-88ba-60712046ad6e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:48.743530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 410/804] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:48.743530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf55a09-5a70-4d78-88ba-60712046ad6e None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:48.837518 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:48.862047 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975970b-819b-4e53-9263-211fe710459b None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:48.917210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e484c13-ecd2-4ff0-b681-dab0f50ab920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:48.918162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e484c13-ecd2-4ff0-b681-dab0f50ab920 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:48.918651 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e484c13-ecd2-4ff0-b681-dab0f50ab920 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:48.919127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e484c13-ecd2-4ff0-b681-dab0f50ab920 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:48.921588 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf55a09-5a70-4d78-88ba-60712046ad6e None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:48.934139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e484c13-ecd2-4ff0-b681-dab0f50ab920 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:48.936971 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e484c13-ecd2-4ff0-b681-dab0f50ab920 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-6e484c13-ecd2-4ff0-b681-dab0f50ab920 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:48.956854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:48.958624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:48.960470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:48.960470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:48.960470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:48.960470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:48.960470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:48.960470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:48.960470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:48.960470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:48.960470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 395/805] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:48.960470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3df578-4c44-4c92-a908-e10a1c668025 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:48.960470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3df578-4c44-4c92-a908-e10a1c668025 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:48.961649 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3df578-4c44-4c92-a908-e10a1c668025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:48.981093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-3c3df578-4c44-4c92-a908-e10a1c668025 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:48.982528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3c3df578-4c44-4c92-a908-e10a1c668025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:48.984101 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e59ac-e75c-45b5-8e31-8161caea29b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:48.985683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e59ac-e75c-45b5-8e31-8161caea29b9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:48.986161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e59ac-e75c-45b5-8e31-8161caea29b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:48.986819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e59ac-e75c-45b5-8e31-8161caea29b9 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:48.987954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e59ac-e75c-45b5-8e31-8161caea29b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:48.988494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e59ac-e75c-45b5-8e31-8161caea29b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:48.990966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e59ac-e75c-45b5-8e31-8161caea29b9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:49.008382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 411/806] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:41:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:49.010025 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e59ac-e75c-45b5-8e31-8161caea29b9 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:49.058277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:49.115506 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 396/807] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:41:49.120712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3df578-4c44-4c92-a908-e10a1c668025 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:49.177860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e59ac-e75c-45b5-8e31-8161caea29b9 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:49.236660 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:49.241816 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a120534-33c0-4d84-96f3-6a2f6f09424f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:49.242682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a120534-33c0-4d84-96f3-6a2f6f09424f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:49.242910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a120534-33c0-4d84-96f3-6a2f6f09424f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:49.243183 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a120534-33c0-4d84-96f3-6a2f6f09424f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:49.262237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a120534-33c0-4d84-96f3-6a2f6f09424f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:49.268882 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a120534-33c0-4d84-96f3-6a2f6f09424f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:49.280917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3df578-4c44-4c92-a908-e10a1c668025 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-8a120534-33c0-4d84-96f3-6a2f6f09424f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:49.291407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:49.293417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:49.295739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:49.295739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:49.295739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:49.295739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:49.297874 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 412/808] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:49.300113 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a120534-33c0-4d84-96f3-6a2f6f09424f None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:49.333027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641f127-d740-40ed-a02d-1a86db972f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:49.333789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641f127-d740-40ed-a02d-1a86db972f69 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:49.334202 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641f127-d740-40ed-a02d-1a86db972f69 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:49.334492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641f127-d740-40ed-a02d-1a86db972f69 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:49.335358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641f127-d740-40ed-a02d-1a86db972f69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:49.335717 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641f127-d740-40ed-a02d-1a86db972f69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:49.337310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641f127-d740-40ed-a02d-1a86db972f69 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:49.356817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 397/809] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:49.358049 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641f127-d740-40ed-a02d-1a86db972f69 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:49.448387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:49.484518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a120534-33c0-4d84-96f3-6a2f6f09424f None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:49.533853 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641f127-d740-40ed-a02d-1a86db972f69 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:49.543154 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f533260-9dbb-4d76-af43-4b6ae56bf767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:49.544336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f533260-9dbb-4d76-af43-4b6ae56bf767 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:49.544856 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f533260-9dbb-4d76-af43-4b6ae56bf767 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:49.545500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f533260-9dbb-4d76-af43-4b6ae56bf767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:49.565475 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f533260-9dbb-4d76-af43-4b6ae56bf767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:49.568021 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f533260-9dbb-4d76-af43-4b6ae56bf767 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-6f533260-9dbb-4d76-af43-4b6ae56bf767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:49.593211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:49.595804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:49.597828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:49.597828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:49.597828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:49.597828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:49.603495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 413/810] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:49.605126 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f533260-9dbb-4d76-af43-4b6ae56bf767 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:49.607175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8274b78-c1b0-45d1-a47e-5b6b35c268b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:49.608236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8274b78-c1b0-45d1-a47e-5b6b35c268b8 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:49.608685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8274b78-c1b0-45d1-a47e-5b6b35c268b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:49.609580 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8274b78-c1b0-45d1-a47e-5b6b35c268b8 None admin] PATH_INFO: `/v3/services/network` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:49.610725 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8274b78-c1b0-45d1-a47e-5b6b35c268b8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:49.613104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8274b78-c1b0-45d1-a47e-5b6b35c268b8 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:49.616024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-e8274b78-c1b0-45d1-a47e-5b6b35c268b8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:49.624968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:49.630058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:49.631126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 398/811] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:49 2022] GET /identity/v3/services/network => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:49.632712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8274b78-c1b0-45d1-a47e-5b6b35c268b8 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:49.784451 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8274b78-c1b0-45d1-a47e-5b6b35c268b8 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:49.813590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f533260-9dbb-4d76-af43-4b6ae56bf767 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:49.846950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b99eeb-87fe-41d2-bae6-2443686154bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:49.847973 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b99eeb-87fe-41d2-bae6-2443686154bc None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:49.848443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b99eeb-87fe-41d2-bae6-2443686154bc None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:49.848918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b99eeb-87fe-41d2-bae6-2443686154bc None admin] PATH_INFO: `/v3/projects/a9b328041da24cedbc801b271d6e0419/users/b6cea7b5ba4a4acabe8a26837cd97310/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:49.852317 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b99eeb-87fe-41d2-bae6-2443686154bc None admin] RBAC: Authorizing `identity:create_grant(project_id=a9b328041da24cedbc801b271d6e0419, user_id=b6cea7b5ba4a4acabe8a26837cd97310, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:49.854660 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b99eeb-87fe-41d2-bae6-2443686154bc None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:49.868917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:49.875605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657f8e5-05a1-4bb8-9331-8f4ad3b0a8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:49.876287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657f8e5-05a1-4bb8-9331-8f4ad3b0a8ad None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:49.876769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657f8e5-05a1-4bb8-9331-8f4ad3b0a8ad None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:49.877112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657f8e5-05a1-4bb8-9331-8f4ad3b0a8ad None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:49.877828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657f8e5-05a1-4bb8-9331-8f4ad3b0a8ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:49.878135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657f8e5-05a1-4bb8-9331-8f4ad3b0a8ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:49.879578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657f8e5-05a1-4bb8-9331-8f4ad3b0a8ad None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:49.893573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 414/812] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:49.894525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657f8e5-05a1-4bb8-9331-8f4ad3b0a8ad None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:49.932122 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 399/813] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:41:49 2022] PUT /identity/v3/projects/a9b328041da24cedbc801b271d6e0419/users/b6cea7b5ba4a4acabe8a26837cd97310/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:41:49.934013 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b99eeb-87fe-41d2-bae6-2443686154bc None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:50.060246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:50.086839 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657f8e5-05a1-4bb8-9331-8f4ad3b0a8ad None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:50.134973 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b99eeb-87fe-41d2-bae6-2443686154bc None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:50.158457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c17db-8ae5-4c7d-a5ec-43e68bb0a63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:50.159964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c17db-8ae5-4c7d-a5ec-43e68bb0a63f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:50.160603 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c17db-8ae5-4c7d-a5ec-43e68bb0a63f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:50.161174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c17db-8ae5-4c7d-a5ec-43e68bb0a63f None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:50.162732 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c17db-8ae5-4c7d-a5ec-43e68bb0a63f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:50.163404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c17db-8ae5-4c7d-a5ec-43e68bb0a63f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:50.166365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c17db-8ae5-4c7d-a5ec-43e68bb0a63f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:50.193128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 415/814] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:50.195140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c17db-8ae5-4c7d-a5ec-43e68bb0a63f None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:50.218851 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9555a88-6be7-4a1c-b72c-0696e0331660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:50.219825 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9555a88-6be7-4a1c-b72c-0696e0331660 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:50.220204 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9555a88-6be7-4a1c-b72c-0696e0331660 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:50.220547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9555a88-6be7-4a1c-b72c-0696e0331660 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:50.221704 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9555a88-6be7-4a1c-b72c-0696e0331660 None admin] RBAC: Adding query filter params (name=network) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:50.222176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9555a88-6be7-4a1c-b72c-0696e0331660 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:50.224054 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9555a88-6be7-4a1c-b72c-0696e0331660 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:50.233989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 400/815] 10.222.0.94 () {56 vars in 1112 bytes} [Tue Jul 26 14:41:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:50.236375 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9555a88-6be7-4a1c-b72c-0696e0331660 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:50.295626 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:50.393311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c17db-8ae5-4c7d-a5ec-43e68bb0a63f None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:50.417985 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9555a88-6be7-4a1c-b72c-0696e0331660 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:50.439193 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849e110-160a-4d8d-81d5-20f8c7c6e0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:50.440137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849e110-160a-4d8d-81d5-20f8c7c6e0df None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:50.440459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849e110-160a-4d8d-81d5-20f8c7c6e0df None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:50.440750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849e110-160a-4d8d-81d5-20f8c7c6e0df None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:50.450823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849e110-160a-4d8d-81d5-20f8c7c6e0df None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:50.452507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849e110-160a-4d8d-81d5-20f8c7c6e0df None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:50.453283 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-5849e110-160a-4d8d-81d5-20f8c7c6e0df None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:50.466712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:50.473069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:50.475037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:50.475037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:50.475037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:50.475037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:50.475037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:50.475037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:50.475037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:50.475037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:50.475037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:41:50.475037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:50.475037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 416/816] 10.222.0.94 () {56 vars in 1117 bytes} [Tue Jul 26 14:41:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:50.475037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849e110-160a-4d8d-81d5-20f8c7c6e0df None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:50.481984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7100d3d-4a2c-46bf-b0a7-f34572d0a6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:50.481984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7100d3d-4a2c-46bf-b0a7-f34572d0a6ec None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:50.481984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7100d3d-4a2c-46bf-b0a7-f34572d0a6ec None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:50.481984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7100d3d-4a2c-46bf-b0a7-f34572d0a6ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:50.494116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7100d3d-4a2c-46bf-b0a7-f34572d0a6ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:50.496052 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7100d3d-4a2c-46bf-b0a7-f34572d0a6ec None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-e7100d3d-4a2c-46bf-b0a7-f34572d0a6ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:50.516116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:50.519215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:50.523058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:50.523058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:50.523058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:50.523058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:50.523058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:50.523058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:50.523058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:50.523058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:50.523058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:41:50.523058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:50.524935 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 401/817] 10.222.0.94 () {56 vars in 1093 bytes} [Tue Jul 26 14:41:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:50.531830 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7100d3d-4a2c-46bf-b0a7-f34572d0a6ec None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:50.612758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849e110-160a-4d8d-81d5-20f8c7c6e0df None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:50.627890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:50.674881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6f8aa-90c6-46d0-9cae-adec249701c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:50.675856 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6f8aa-90c6-46d0-9cae-adec249701c9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:50.676339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6f8aa-90c6-46d0-9cae-adec249701c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:50.676821 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6f8aa-90c6-46d0-9cae-adec249701c9 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:50.678085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6f8aa-90c6-46d0-9cae-adec249701c9 None admin] RBAC: Adding query filter params (type=network) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:50.678614 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6f8aa-90c6-46d0-9cae-adec249701c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:50.681304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6f8aa-90c6-46d0-9cae-adec249701c9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:50.696137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 417/818] 10.222.0.94 () {56 vars in 1112 bytes} [Tue Jul 26 14:41:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:50.696137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6f8aa-90c6-46d0-9cae-adec249701c9 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:50.700349 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7100d3d-4a2c-46bf-b0a7-f34572d0a6ec None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:50.760388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c95042-70be-4155-ba9e-ada00475236f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:50.761341 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c95042-70be-4155-ba9e-ada00475236f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:50.761829 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c95042-70be-4155-ba9e-ada00475236f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:50.762285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c95042-70be-4155-ba9e-ada00475236f None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:50.764044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c95042-70be-4155-ba9e-ada00475236f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:50.764565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c95042-70be-4155-ba9e-ada00475236f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:50.766699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c95042-70be-4155-ba9e-ada00475236f None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:50.795437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 402/819] 10.222.0.94 () {56 vars in 1149 bytes} [Tue Jul 26 14:41:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:50.796790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c95042-70be-4155-ba9e-ada00475236f None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:50.853975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6f8aa-90c6-46d0-9cae-adec249701c9 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:50.865819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:50.897807 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54caff-a14c-44c2-8f9a-b761ec5153f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:50.899016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54caff-a14c-44c2-8f9a-b761ec5153f5 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:50.899728 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54caff-a14c-44c2-8f9a-b761ec5153f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:50.900312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54caff-a14c-44c2-8f9a-b761ec5153f5 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:50.901447 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54caff-a14c-44c2-8f9a-b761ec5153f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:50.902100 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54caff-a14c-44c2-8f9a-b761ec5153f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:50.903936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54caff-a14c-44c2-8f9a-b761ec5153f5 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:50.939891 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 418/820] 10.222.0.94 () {56 vars in 1133 bytes} [Tue Jul 26 14:41:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:50.941878 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54caff-a14c-44c2-8f9a-b761ec5153f5 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:50.990116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c95042-70be-4155-ba9e-ada00475236f None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:51.008117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:51.048494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ac124-4bf9-4a15-a17d-c6c388cc7817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:51.049298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ac124-4bf9-4a15-a17d-c6c388cc7817 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:51.049790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ac124-4bf9-4a15-a17d-c6c388cc7817 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:51.050207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ac124-4bf9-4a15-a17d-c6c388cc7817 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:51.051225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ac124-4bf9-4a15-a17d-c6c388cc7817 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:51.052797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ac124-4bf9-4a15-a17d-c6c388cc7817 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:51.143504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 403/821] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 14:41:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:41:51.143504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ac124-4bf9-4a15-a17d-c6c388cc7817 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:51.168318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54caff-a14c-44c2-8f9a-b761ec5153f5 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:51.253599 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61cd16e-c82a-4c14-bbe8-09109e07a823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:51.254047 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cd16e-c82a-4c14-bbe8-09109e07a823 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:51.254324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cd16e-c82a-4c14-bbe8-09109e07a823 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:51.254846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cd16e-c82a-4c14-bbe8-09109e07a823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:51.272894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61cd16e-c82a-4c14-bbe8-09109e07a823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:51.274785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61cd16e-c82a-4c14-bbe8-09109e07a823 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-e61cd16e-c82a-4c14-bbe8-09109e07a823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:51.295000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:51.296886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:51.301362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:51.301362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:51.301362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:51.301362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:51.302490 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 419/822] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:51.304604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca637-8577-47a8-b987-cf2d15cff0ed None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:51.304959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca637-8577-47a8-b987-cf2d15cff0ed None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:51.305192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ca637-8577-47a8-b987-cf2d15cff0ed None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:51.308153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 420/823] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:51.309298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ca637-8577-47a8-b987-cf2d15cff0ed None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:51.380039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:51.395174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ac124-4bf9-4a15-a17d-c6c388cc7817 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:51.430330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:51.466794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ca637-8577-47a8-b987-cf2d15cff0ed None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:51.467944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd830b-3a9f-453f-ac6f-3d25371653fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:51.469711 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd830b-3a9f-453f-ac6f-3d25371653fd None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:51.472683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd830b-3a9f-453f-ac6f-3d25371653fd None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:51.473194 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd830b-3a9f-453f-ac6f-3d25371653fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:51.528138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8b502-f123-4162-935f-2ae9b65135d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:51.529182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8b502-f123-4162-935f-2ae9b65135d9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:51.529661 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8b502-f123-4162-935f-2ae9b65135d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:51.530122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8b502-f123-4162-935f-2ae9b65135d9 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:51.531376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8b502-f123-4162-935f-2ae9b65135d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:51.532456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8b502-f123-4162-935f-2ae9b65135d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:51.535598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8b502-f123-4162-935f-2ae9b65135d9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:51.562272 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 421/824] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:51.564444 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48233ec3-ea61-497a-822d-db7689fb3df9 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:51.564956 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48233ec3-ea61-497a-822d-db7689fb3df9 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:51.565448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48233ec3-ea61-497a-822d-db7689fb3df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:51.602461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-48233ec3-ea61-497a-822d-db7689fb3df9 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:51.603729 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-48233ec3-ea61-497a-822d-db7689fb3df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:51.617463 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd830b-3a9f-453f-ac6f-3d25371653fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:51.618913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd830b-3a9f-453f-ac6f-3d25371653fd None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:51.722433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 422/825] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:41:51.722433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48233ec3-ea61-497a-822d-db7689fb3df9 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:51.780690 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:51.800867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 404/826] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:51 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:41:51.803260 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd830b-3a9f-453f-ac6f-3d25371653fd None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:51.880747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:51.939919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48233ec3-ea61-497a-822d-db7689fb3df9 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:51.964682 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd830b-3a9f-453f-ac6f-3d25371653fd None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:52.001472 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83e9a6-d0b1-4bf7-8bc9-2059dfb1dbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:52.002505 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83e9a6-d0b1-4bf7-8bc9-2059dfb1dbac None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:52.002966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83e9a6-d0b1-4bf7-8bc9-2059dfb1dbac None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:52.003414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83e9a6-d0b1-4bf7-8bc9-2059dfb1dbac None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:52.013791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83e9a6-d0b1-4bf7-8bc9-2059dfb1dbac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:52.016987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83e9a6-d0b1-4bf7-8bc9-2059dfb1dbac None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:52.031199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1bc8f-e5a6-4f7c-b5c7-8f287e953fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:52.032938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1bc8f-e5a6-4f7c-b5c7-8f287e953fd5 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:52.033379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1bc8f-e5a6-4f7c-b5c7-8f287e953fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:52.033842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1bc8f-e5a6-4f7c-b5c7-8f287e953fd5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:52.035256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1bc8f-e5a6-4f7c-b5c7-8f287e953fd5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:52.038036 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1bc8f-e5a6-4f7c-b5c7-8f287e953fd5 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-3a83e9a6-d0b1-4bf7-8bc9-2059dfb1dbac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:52.040150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:52.041851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:52.043645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:52.043645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:52.043645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:52.043645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:52.043645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:52.043645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:52.043645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:52.043645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:52.043645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 423/827] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:52.045427 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83e9a6-d0b1-4bf7-8bc9-2059dfb1dbac None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-b0e1bc8f-e5a6-4f7c-b5c7-8f287e953fd5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:52.070906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:52.073242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:52.075432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:52.075432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:52.075432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:52.075432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:52.075432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:52.075432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:52.075432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:52.075432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:52.075432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:52.075432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:52.075432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:41:52.075432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:52.075432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:41:52.075432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:52.078265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 405/828] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:41:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:52.078265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1bc8f-e5a6-4f7c-b5c7-8f287e953fd5 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:52.216958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:52.261637 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83e9a6-d0b1-4bf7-8bc9-2059dfb1dbac None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:52.309501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1bc8f-e5a6-4f7c-b5c7-8f287e953fd5 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:52.341971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:52.354159 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49074fb-e0a4-4093-a3ba-b22e0800e1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:52.355446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49074fb-e0a4-4093-a3ba-b22e0800e1ab None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:52.356063 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49074fb-e0a4-4093-a3ba-b22e0800e1ab None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:52.356773 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49074fb-e0a4-4093-a3ba-b22e0800e1ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:52.363256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a15ee-75b0-42f7-ad7d-adf8a17526bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:52.368768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a15ee-75b0-42f7-ad7d-adf8a17526bf None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:52.368768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a15ee-75b0-42f7-ad7d-adf8a17526bf None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:52.368768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a15ee-75b0-42f7-ad7d-adf8a17526bf None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:52.368768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a15ee-75b0-42f7-ad7d-adf8a17526bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:52.368768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a15ee-75b0-42f7-ad7d-adf8a17526bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:52.368768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a15ee-75b0-42f7-ad7d-adf8a17526bf None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:52.377696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49074fb-e0a4-4093-a3ba-b22e0800e1ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:52.380619 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49074fb-e0a4-4093-a3ba-b22e0800e1ab None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:52.386696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 406/829] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:52.387982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a15ee-75b0-42f7-ad7d-adf8a17526bf None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-c49074fb-e0a4-4093-a3ba-b22e0800e1ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:52.411550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:52.413103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:52.414149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:52.414149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:52.414149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:52.414149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:52.414149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 424/830] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:52.415242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49074fb-e0a4-4093-a3ba-b22e0800e1ab None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:52.538581 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a15ee-75b0-42f7-ad7d-adf8a17526bf None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:52.564886 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:52.582577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49074fb-e0a4-4093-a3ba-b22e0800e1ab None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:52.583440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c47ed-b513-4d62-94cb-10b43073d28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:52.584192 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c47ed-b513-4d62-94cb-10b43073d28c None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:52.584596 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c47ed-b513-4d62-94cb-10b43073d28c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:52.584936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c47ed-b513-4d62-94cb-10b43073d28c None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:52.586000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c47ed-b513-4d62-94cb-10b43073d28c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:52.586508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c47ed-b513-4d62-94cb-10b43073d28c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:52.588220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c47ed-b513-4d62-94cb-10b43073d28c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:52.599753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 407/831] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:41:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:52.599753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c47ed-b513-4d62-94cb-10b43073d28c None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:52.631865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be9e4a-7113-4daa-a404-496f36177be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:52.632757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be9e4a-7113-4daa-a404-496f36177be2 None admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:52.633119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be9e4a-7113-4daa-a404-496f36177be2 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:52.633455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be9e4a-7113-4daa-a404-496f36177be2 None admin] PATH_INFO: `/v3/projects/a9b328041da24cedbc801b271d6e0419/users/5b24d190536a4953930432287dfbfb9b/roles/170d72cc35f34a249c65a819a18638a4` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:52.740343 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c47ed-b513-4d62-94cb-10b43073d28c None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:52.795361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02e5bf-c4c0-464c-9caa-fb4908eec983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:52.796178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02e5bf-c4c0-464c-9caa-fb4908eec983 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:52.796552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02e5bf-c4c0-464c-9caa-fb4908eec983 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:52.796938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02e5bf-c4c0-464c-9caa-fb4908eec983 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:52.797813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02e5bf-c4c0-464c-9caa-fb4908eec983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:52.798328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02e5bf-c4c0-464c-9caa-fb4908eec983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:52.799912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02e5bf-c4c0-464c-9caa-fb4908eec983 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:52.814727 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 408/832] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:52.816206 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939537a-a072-4ef1-bcf0-1903cdad7d15 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:52.816470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939537a-a072-4ef1-bcf0-1903cdad7d15 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:52.816701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939537a-a072-4ef1-bcf0-1903cdad7d15 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:52.823751 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 409/833] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:52.823751 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939537a-a072-4ef1-bcf0-1903cdad7d15 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:52.837970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be9e4a-7113-4daa-a404-496f36177be2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9b328041da24cedbc801b271d6e0419, user_id=5b24d190536a4953930432287dfbfb9b, role_id=170d72cc35f34a249c65a819a18638a4)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:52.840787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be9e4a-7113-4daa-a404-496f36177be2 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:52.879862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:52.930857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 425/834] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:41:52 2022] PUT /identity/v3/projects/a9b328041da24cedbc801b271d6e0419/users/5b24d190536a4953930432287dfbfb9b/roles/170d72cc35f34a249c65a819a18638a4 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:41:52.931763 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be9e4a-7113-4daa-a404-496f36177be2 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:52.952419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939537a-a072-4ef1-bcf0-1903cdad7d15 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:52.983939 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:53.001915 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1e1e5-6e85-48bf-852d-8d83d24b1896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:53.002544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1e1e5-6e85-48bf-852d-8d83d24b1896 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:53.002874 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1e1e5-6e85-48bf-852d-8d83d24b1896 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:53.003190 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1e1e5-6e85-48bf-852d-8d83d24b1896 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:53.005006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1e1e5-6e85-48bf-852d-8d83d24b1896 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:53.005417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1e1e5-6e85-48bf-852d-8d83d24b1896 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:53.007078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1e1e5-6e85-48bf-852d-8d83d24b1896 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:53.014376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 410/835] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:41:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:53.016151 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f1f4f-84bd-4c62-bdd9-ea7d7236a05c None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:53.016591 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f1f4f-84bd-4c62-bdd9-ea7d7236a05c None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:53.016591 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f1f4f-84bd-4c62-bdd9-ea7d7236a05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:53.065623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-082f1f4f-84bd-4c62-bdd9-ea7d7236a05c None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:53.068850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-082f1f4f-84bd-4c62-bdd9-ea7d7236a05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:53.148253 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be9e4a-7113-4daa-a404-496f36177be2 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:53.200097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 411/836] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:41:53.226860 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f1f4f-84bd-4c62-bdd9-ea7d7236a05c None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:53.227597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740511a-7200-4448-9ec1-394fd90398a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:53.228119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740511a-7200-4448-9ec1-394fd90398a8 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:53.228568 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740511a-7200-4448-9ec1-394fd90398a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:53.229008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740511a-7200-4448-9ec1-394fd90398a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:53.242996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740511a-7200-4448-9ec1-394fd90398a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:53.245615 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740511a-7200-4448-9ec1-394fd90398a8 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-c740511a-7200-4448-9ec1-394fd90398a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:53.273238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:53.275243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:53.277230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:53.277230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:53.277230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:53.277230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:53.277230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:53.277230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:53.277230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:53.277230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:53.277230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 426/837] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:53.285896 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740511a-7200-4448-9ec1-394fd90398a8 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:53.291373 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:53.411506 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f1f4f-84bd-4c62-bdd9-ea7d7236a05c None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:53.469249 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:53.486001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39646693-aef3-43d8-8901-2d7d614fc792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:53.486371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740511a-7200-4448-9ec1-394fd90398a8 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:53.487811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39646693-aef3-43d8-8901-2d7d614fc792 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:53.488520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39646693-aef3-43d8-8901-2d7d614fc792 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:53.489156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39646693-aef3-43d8-8901-2d7d614fc792 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:53.511119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39646693-aef3-43d8-8901-2d7d614fc792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-39646693-aef3-43d8-8901-2d7d614fc792 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:53.522909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:53.525029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:53.526730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:53.526730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:53.526730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:53.526730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:53.526730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:53.526730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:53.526730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:53.526730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:53.526730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:53.526730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:53.526730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:41:53.526730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:53.531669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 412/838] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:41:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:41:53.541891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39646693-aef3-43d8-8901-2d7d614fc792 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:53.553311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0ba49-b458-4022-9ad9-72982e9a962b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:53.554300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0ba49-b458-4022-9ad9-72982e9a962b None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:53.554766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0ba49-b458-4022-9ad9-72982e9a962b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:53.555228 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0ba49-b458-4022-9ad9-72982e9a962b None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:53.556809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0ba49-b458-4022-9ad9-72982e9a962b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:53.557334 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0ba49-b458-4022-9ad9-72982e9a962b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:53.560137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0ba49-b458-4022-9ad9-72982e9a962b None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:53.591656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 427/839] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:41:53.594739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0ba49-b458-4022-9ad9-72982e9a962b None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:53.684595 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:53.690535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39646693-aef3-43d8-8901-2d7d614fc792 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:53.747764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19009ba5-e57d-4d22-9056-978e5d136bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:53.748572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19009ba5-e57d-4d22-9056-978e5d136bc1 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:53.748934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19009ba5-e57d-4d22-9056-978e5d136bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:53.749285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19009ba5-e57d-4d22-9056-978e5d136bc1 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:53.750318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19009ba5-e57d-4d22-9056-978e5d136bc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:53.750906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19009ba5-e57d-4d22-9056-978e5d136bc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:53.753238 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19009ba5-e57d-4d22-9056-978e5d136bc1 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:53.771733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 413/840] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:41:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:53.774206 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19009ba5-e57d-4d22-9056-978e5d136bc1 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:53.804522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0ba49-b458-4022-9ad9-72982e9a962b None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:53.853597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1389d-0b8f-439f-b72a-7be57171b02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:53.854823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1389d-0b8f-439f-b72a-7be57171b02d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:53.855327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1389d-0b8f-439f-b72a-7be57171b02d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:53.856586 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1389d-0b8f-439f-b72a-7be57171b02d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:53.858182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1389d-0b8f-439f-b72a-7be57171b02d None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=6a4278fe995e432d88f4346d6f54218b) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:53.858745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1389d-0b8f-439f-b72a-7be57171b02d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:53.861997 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1389d-0b8f-439f-b72a-7be57171b02d None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:53.873191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 428/841] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:53 2022] GET /identity/v3/role_assignments?user.id=6a4278fe995e432d88f4346d6f54218b&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:41:53.938886 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19009ba5-e57d-4d22-9056-978e5d136bc1 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:53.991426 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178f954-998d-42b3-bb47-44c78e5886cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:53.992804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178f954-998d-42b3-bb47-44c78e5886cd None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:53.993505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178f954-998d-42b3-bb47-44c78e5886cd None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:53.994063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178f954-998d-42b3-bb47-44c78e5886cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:54.011087 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178f954-998d-42b3-bb47-44c78e5886cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:54.014159 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178f954-998d-42b3-bb47-44c78e5886cd None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-2178f954-998d-42b3-bb47-44c78e5886cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:54.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:54.041032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:54.042064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:54.042064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:54.042064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:54.042064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:54.044825 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 414/842] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:54.052685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1389d-0b8f-439f-b72a-7be57171b02d None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:54.080880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:54.191881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1389d-0b8f-439f-b72a-7be57171b02d None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:54.239034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d2c00-3f89-4d5e-b969-e5eee2962a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:54.240177 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d2c00-3f89-4d5e-b969-e5eee2962a9b None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:54.240747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d2c00-3f89-4d5e-b969-e5eee2962a9b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:54.241283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d2c00-3f89-4d5e-b969-e5eee2962a9b None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:54.242415 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d2c00-3f89-4d5e-b969-e5eee2962a9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:54.242996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d2c00-3f89-4d5e-b969-e5eee2962a9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:54.244854 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d2c00-3f89-4d5e-b969-e5eee2962a9b None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:54.268028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 429/843] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:54.272476 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178f954-998d-42b3-bb47-44c78e5886cd None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:54.309051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:54.446328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178f954-998d-42b3-bb47-44c78e5886cd None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:54.503015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ba9e2-c2c9-4d2e-a0f3-955f5b2aa79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:54.503749 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ba9e2-c2c9-4d2e-a0f3-955f5b2aa79c None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:54.504053 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ba9e2-c2c9-4d2e-a0f3-955f5b2aa79c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:54.504409 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ba9e2-c2c9-4d2e-a0f3-955f5b2aa79c None admin] PATH_INFO: `/v3/users/swift` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:54.514821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ba9e2-c2c9-4d2e-a0f3-955f5b2aa79c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:54.516559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ba9e2-c2c9-4d2e-a0f3-955f5b2aa79c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-118ba9e2-c2c9-4d2e-a0f3-955f5b2aa79c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:54.531130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:54.533220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:54.534951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:54.534951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:54.534951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:54.534951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:54.534951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:54.534951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:54.534951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:54.534951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:54.534951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 14:41:54.534951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:54.534951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 415/844] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:41:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:41:54.540970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d2c00-3f89-4d5e-b969-e5eee2962a9b None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:54.646432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:54.676883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d2c00-3f89-4d5e-b969-e5eee2962a9b None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:54.720903 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e971a-a7c8-444b-b01f-3dc3396e34ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:54.721514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e971a-a7c8-444b-b01f-3dc3396e34ac None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:54.721784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e971a-a7c8-444b-b01f-3dc3396e34ac None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:54.722026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e971a-a7c8-444b-b01f-3dc3396e34ac None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:54.722875 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e971a-a7c8-444b-b01f-3dc3396e34ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:54.723179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e971a-a7c8-444b-b01f-3dc3396e34ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:54.724811 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e971a-a7c8-444b-b01f-3dc3396e34ac None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:54.748597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 430/845] 10.222.0.94 () {56 vars in 1135 bytes} [Tue Jul 26 14:41:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:41:54.754734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ba9e2-c2c9-4d2e-a0f3-955f5b2aa79c None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:54.853097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:54.895307 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ba9e2-c2c9-4d2e-a0f3-955f5b2aa79c None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:54.947403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cde22c-b6b5-4e6a-bcc1-dca34db3cb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:54.948152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cde22c-b6b5-4e6a-bcc1-dca34db3cb7e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:54.948437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cde22c-b6b5-4e6a-bcc1-dca34db3cb7e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:54.948785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cde22c-b6b5-4e6a-bcc1-dca34db3cb7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:55.059167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cde22c-b6b5-4e6a-bcc1-dca34db3cb7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:55.060736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cde22c-b6b5-4e6a-bcc1-dca34db3cb7e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:55.160964 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:55.185083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 416/846] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:54 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 14:41:55.191906 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e971a-a7c8-444b-b01f-3dc3396e34ac None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:55.403378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e971a-a7c8-444b-b01f-3dc3396e34ac None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:55.474041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adb46a-13c6-4053-ac4e-112157b52a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:55.475177 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adb46a-13c6-4053-ac4e-112157b52a43 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:55.475696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adb46a-13c6-4053-ac4e-112157b52a43 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:55.476173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adb46a-13c6-4053-ac4e-112157b52a43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:55.479932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:55.498162 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79adb46a-13c6-4053-ac4e-112157b52a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:55.501308 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79adb46a-13c6-4053-ac4e-112157b52a43 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-79adb46a-13c6-4053-ac4e-112157b52a43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:55.533263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:55.535283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:55.538137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:55.538137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:55.538137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:55.538137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:55.538137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 431/847] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:41:55.541780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cde22c-b6b5-4e6a-bcc1-dca34db3cb7e None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:55.674338 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cde22c-b6b5-4e6a-bcc1-dca34db3cb7e None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:55.717940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae53ca5-e311-4b9f-90b1-9d0bb3c582ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:55.718576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae53ca5-e311-4b9f-90b1-9d0bb3c582ee None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:55.719026 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae53ca5-e311-4b9f-90b1-9d0bb3c582ee None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:55.719352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae53ca5-e311-4b9f-90b1-9d0bb3c582ee None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:55.720226 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae53ca5-e311-4b9f-90b1-9d0bb3c582ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:55.720541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae53ca5-e311-4b9f-90b1-9d0bb3c582ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:55.721967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae53ca5-e311-4b9f-90b1-9d0bb3c582ee None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:55.738975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 417/848] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:55.745798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adb46a-13c6-4053-ac4e-112157b52a43 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:55.946912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adb46a-13c6-4053-ac4e-112157b52a43 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:55.991082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984053c8-4f39-45d4-a7d6-55b61e84f3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:55.992128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984053c8-4f39-45d4-a7d6-55b61e84f3a6 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:55.992591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984053c8-4f39-45d4-a7d6-55b61e84f3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:55.993043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984053c8-4f39-45d4-a7d6-55b61e84f3a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:56.002516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984053c8-4f39-45d4-a7d6-55b61e84f3a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:56.005171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984053c8-4f39-45d4-a7d6-55b61e84f3a6 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:56.021752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdba4ff-7932-468d-bfd7-b7b65094e9b5 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:56.022240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdba4ff-7932-468d-bfd7-b7b65094e9b5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:56.022776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdba4ff-7932-468d-bfd7-b7b65094e9b5 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:56.025904 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 418/849] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-984053c8-4f39-45d4-a7d6-55b61e84f3a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:41:56.028145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:56.029082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:56.029993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:41:56.029993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:41:56.029993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:41:56.029993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:41:56.029993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:41:56.029993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:41:56.029993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:41:56.029993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:56.033166 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 432/850] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:41:56.033324 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c46a6-c924-49e8-a5d3-0d69bc4563e4 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:56.033534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c46a6-c924-49e8-a5d3-0d69bc4563e4 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:56.039393 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c46a6-c924-49e8-a5d3-0d69bc4563e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:56.041311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984053c8-4f39-45d4-a7d6-55b61e84f3a6 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:56.058994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-265c46a6-c924-49e8-a5d3-0d69bc4563e4 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:56.060498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-265c46a6-c924-49e8-a5d3-0d69bc4563e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:56.112757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:56.144583 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:56.192998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 419/851] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:41:56.195871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c7060-e287-4808-b632-1a6dfb4a05ff None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:56.196369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c7060-e287-4808-b632-1a6dfb4a05ff None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:56.196868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c7060-e287-4808-b632-1a6dfb4a05ff None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:56.200055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 420/852] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:56.208850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd2144-eaef-4e5b-b349-e25df0f7a200 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:56.209353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd2144-eaef-4e5b-b349-e25df0f7a200 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:56.209852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd2144-eaef-4e5b-b349-e25df0f7a200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:56.246358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-78cd2144-eaef-4e5b-b349-e25df0f7a200 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:56.246358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-78cd2144-eaef-4e5b-b349-e25df0f7a200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:56.270276 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984053c8-4f39-45d4-a7d6-55b61e84f3a6 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:56.320240 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d137ed4-ec3f-4d31-89d6-f54416ce1b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:56.320886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d137ed4-ec3f-4d31-89d6-f54416ce1b81 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:56.321151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d137ed4-ec3f-4d31-89d6-f54416ce1b81 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:56.321425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d137ed4-ec3f-4d31-89d6-f54416ce1b81 None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:56.322199 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d137ed4-ec3f-4d31-89d6-f54416ce1b81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:56.322861 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d137ed4-ec3f-4d31-89d6-f54416ce1b81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:56.324589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d137ed4-ec3f-4d31-89d6-f54416ce1b81 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:56.344952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 433/853] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:41:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:41:56.345659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d137ed4-ec3f-4d31-89d6-f54416ce1b81 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:56.374729 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 421/854] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:41:56.380586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd2144-eaef-4e5b-b349-e25df0f7a200 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:56.528899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d137ed4-ec3f-4d31-89d6-f54416ce1b81 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:56.543983 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:56.578455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:56.591601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1636aa-19c4-468b-a4e8-c889956a9fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:56.592443 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1636aa-19c4-468b-a4e8-c889956a9fd8 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:56.592846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1636aa-19c4-468b-a4e8-c889956a9fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:56.593248 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1636aa-19c4-468b-a4e8-c889956a9fd8 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:56.593505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd2144-eaef-4e5b-b349-e25df0f7a200 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:56.595395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1636aa-19c4-468b-a4e8-c889956a9fd8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:56.597536 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1636aa-19c4-468b-a4e8-c889956a9fd8 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:56.647767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5eabcb-3d4a-48e0-bdd4-281cd00ed16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:56.648818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5eabcb-3d4a-48e0-bdd4-281cd00ed16d None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:56.649304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5eabcb-3d4a-48e0-bdd4-281cd00ed16d None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:56.649791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5eabcb-3d4a-48e0-bdd4-281cd00ed16d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:56.651688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5eabcb-3d4a-48e0-bdd4-281cd00ed16d None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=b6cea7b5ba4a4acabe8a26837cd97310) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:56.652247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5eabcb-3d4a-48e0-bdd4-281cd00ed16d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:56.655289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5eabcb-3d4a-48e0-bdd4-281cd00ed16d None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:56.668518 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 422/855] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:41:56 2022] GET /identity/v3/role_assignments?user.id=b6cea7b5ba4a4acabe8a26837cd97310&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:56.669950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5eabcb-3d4a-48e0-bdd4-281cd00ed16d None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:56.755666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 434/856] 10.222.0.94 () {60 vars in 1141 bytes} [Tue Jul 26 14:41:56 2022] POST /identity/v3/services => generated 251 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:41:56.854995 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5eabcb-3d4a-48e0-bdd4-281cd00ed16d None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:56.925827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1370e-c788-47d5-9a92-153bda4c520e None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:56.926182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1370e-c788-47d5-9a92-153bda4c520e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:56.926462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1370e-c788-47d5-9a92-153bda4c520e None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:56.928479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 435/857] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:56.935446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc96ae-9e99-4a0b-86b6-33e3ba18b942 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:56.936265 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc96ae-9e99-4a0b-86b6-33e3ba18b942 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:56.936742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc96ae-9e99-4a0b-86b6-33e3ba18b942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:56.936832 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc1ef0-5b0d-4d89-834b-8220231d710d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:56.937780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc1ef0-5b0d-4d89-834b-8220231d710d None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:56.938263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc1ef0-5b0d-4d89-834b-8220231d710d None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:56.938970 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc1ef0-5b0d-4d89-834b-8220231d710d None admin] PATH_INFO: `/v3/roles/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:56.961619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc1ef0-5b0d-4d89-834b-8220231d710d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:41:56.964054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-50cc96ae-9e99-4a0b-86b6-33e3ba18b942 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:56.966596 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-50cc96ae-9e99-4a0b-86b6-33e3ba18b942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-9bcc1ef0-5b0d-4d89-834b-8220231d710d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:41:56.967300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:56.969619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:56.971440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:56.971440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:56.971440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:56.971440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:56.971440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:41:56.971440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:41:56.971440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:41:56.971440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:41:56.971440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:41:56.971440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:41:56.971440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:41:56.971440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:56.971440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 423/858] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:41:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:56.982901 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc1ef0-5b0d-4d89-834b-8220231d710d None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:57.004030 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:57.099682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 436/859] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:56 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:41:57.140148 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc96ae-9e99-4a0b-86b6-33e3ba18b942 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:57.149510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc1ef0-5b0d-4d89-834b-8220231d710d None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:57.207147 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edd1a4-6e6c-410f-904f-80edb038ba40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:57.208219 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edd1a4-6e6c-410f-904f-80edb038ba40 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:57.208717 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edd1a4-6e6c-410f-904f-80edb038ba40 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:57.209188 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edd1a4-6e6c-410f-904f-80edb038ba40 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:57.210365 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edd1a4-6e6c-410f-904f-80edb038ba40 None admin] RBAC: Adding query filter params (name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:57.210964 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edd1a4-6e6c-410f-904f-80edb038ba40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:57.212722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edd1a4-6e6c-410f-904f-80edb038ba40 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:57.235677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 424/860] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:41:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:41:57.239126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edd1a4-6e6c-410f-904f-80edb038ba40 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:57.321846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:57.333388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc96ae-9e99-4a0b-86b6-33e3ba18b942 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:57.397112 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:57.408249 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93638ecf-ebce-49d0-8fd5-d7fa625f5424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:57.410185 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93638ecf-ebce-49d0-8fd5-d7fa625f5424 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:57.410901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93638ecf-ebce-49d0-8fd5-d7fa625f5424 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:57.411605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93638ecf-ebce-49d0-8fd5-d7fa625f5424 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:57.412942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93638ecf-ebce-49d0-8fd5-d7fa625f5424 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:57.415307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93638ecf-ebce-49d0-8fd5-d7fa625f5424 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:57.430803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edd1a4-6e6c-410f-904f-80edb038ba40 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-93638ecf-ebce-49d0-8fd5-d7fa625f5424 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:57.435855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:57.437970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:41:57.439894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:57.451815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 437/861] 10.222.0.94 () {56 vars in 1132 bytes} [Tue Jul 26 14:41:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:41:57.456024 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93638ecf-ebce-49d0-8fd5-d7fa625f5424 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:57.477698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189e77f-0d3d-4e37-b7c4-8a7c44cb0b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:57.478467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189e77f-0d3d-4e37-b7c4-8a7c44cb0b88 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:57.478820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189e77f-0d3d-4e37-b7c4-8a7c44cb0b88 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:57.479167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189e77f-0d3d-4e37-b7c4-8a7c44cb0b88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:57.496382 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189e77f-0d3d-4e37-b7c4-8a7c44cb0b88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:57.498595 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189e77f-0d3d-4e37-b7c4-8a7c44cb0b88 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-1189e77f-0d3d-4e37-b7c4-8a7c44cb0b88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:57.520545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:57.522355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:57.524824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:57.524824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:57.524824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:57.524824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:57.527141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 425/862] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:41:57.528603 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189e77f-0d3d-4e37-b7c4-8a7c44cb0b88 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:57.676719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93638ecf-ebce-49d0-8fd5-d7fa625f5424 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:57.717956 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189e77f-0d3d-4e37-b7c4-8a7c44cb0b88 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:57.747529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba1088-c42c-4df4-b3f5-0b53b13a6fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:57.749168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba1088-c42c-4df4-b3f5-0b53b13a6fd0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:57.750003 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba1088-c42c-4df4-b3f5-0b53b13a6fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:57.750732 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba1088-c42c-4df4-b3f5-0b53b13a6fd0 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:57.752339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba1088-c42c-4df4-b3f5-0b53b13a6fd0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:57.753158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba1088-c42c-4df4-b3f5-0b53b13a6fd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:57.756197 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba1088-c42c-4df4-b3f5-0b53b13a6fd0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:57.772098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 438/863] 10.222.0.94 () {56 vars in 1126 bytes} [Tue Jul 26 14:41:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:41:57.773854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:57.778824 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba1088-c42c-4df4-b3f5-0b53b13a6fd0 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:57.792033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b85347-f046-4ebb-b1b9-01fcc6c01092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:57.799392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b85347-f046-4ebb-b1b9-01fcc6c01092 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:57.799392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b85347-f046-4ebb-b1b9-01fcc6c01092 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:57.799392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b85347-f046-4ebb-b1b9-01fcc6c01092 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:57.799392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b85347-f046-4ebb-b1b9-01fcc6c01092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:57.799392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b85347-f046-4ebb-b1b9-01fcc6c01092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:57.799392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b85347-f046-4ebb-b1b9-01fcc6c01092 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:57.825101 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 426/864] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:57.832274 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b85347-f046-4ebb-b1b9-01fcc6c01092 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:57.838330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:57.997404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba1088-c42c-4df4-b3f5-0b53b13a6fd0 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:58.049926 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b85347-f046-4ebb-b1b9-01fcc6c01092 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:58.070128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358b5b2-9992-42dd-afca-4a802a3bfab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:58.071423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358b5b2-9992-42dd-afca-4a802a3bfab3 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:58.072017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358b5b2-9992-42dd-afca-4a802a3bfab3 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:58.072573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358b5b2-9992-42dd-afca-4a802a3bfab3 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:58.073833 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358b5b2-9992-42dd-afca-4a802a3bfab3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:58.074417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358b5b2-9992-42dd-afca-4a802a3bfab3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:58.077381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358b5b2-9992-42dd-afca-4a802a3bfab3 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:58.096304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 439/865] 10.222.0.94 () {56 vars in 1126 bytes} [Tue Jul 26 14:41:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:58.124561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14baa604-c208-4488-8d42-1d4a15791fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:58.126830 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14baa604-c208-4488-8d42-1d4a15791fe0 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:58.127617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14baa604-c208-4488-8d42-1d4a15791fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:58.128309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14baa604-c208-4488-8d42-1d4a15791fe0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:58.145123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14baa604-c208-4488-8d42-1d4a15791fe0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:58.148538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14baa604-c208-4488-8d42-1d4a15791fe0 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-14baa604-c208-4488-8d42-1d4a15791fe0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:41:58.175176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:58.177333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:58.179501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:58.179501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:58.179501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:41:58.179501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:41:58.179501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:41:58.179501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:41:58.179501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:41:58.179501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:41:58.179501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:41:58.179501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:41:58.183706 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 427/866] 10.222.0.94 () {56 vars in 1117 bytes} [Tue Jul 26 14:41:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:58.195557 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358b5b2-9992-42dd-afca-4a802a3bfab3 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:58.384852 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:58.420085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358b5b2-9992-42dd-afca-4a802a3bfab3 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:58.494883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a42ab9-fe3f-4275-a0da-26ba5ab77d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:58.495955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a42ab9-fe3f-4275-a0da-26ba5ab77d80 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:58.496443 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a42ab9-fe3f-4275-a0da-26ba5ab77d80 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:58.496925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a42ab9-fe3f-4275-a0da-26ba5ab77d80 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:58.498614 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a42ab9-fe3f-4275-a0da-26ba5ab77d80 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:58.499036 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a42ab9-fe3f-4275-a0da-26ba5ab77d80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:58.502622 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a42ab9-fe3f-4275-a0da-26ba5ab77d80 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:58.542000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 440/867] 10.222.0.94 () {56 vars in 1149 bytes} [Tue Jul 26 14:41:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:58.552441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14baa604-c208-4488-8d42-1d4a15791fe0 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:58.646331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:58.761243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14baa604-c208-4488-8d42-1d4a15791fe0 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:58.836213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472121e3-a82c-4dce-a1fd-ae23c0a468c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:58.837807 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472121e3-a82c-4dce-a1fd-ae23c0a468c7 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:58.838623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472121e3-a82c-4dce-a1fd-ae23c0a468c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:58.839406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472121e3-a82c-4dce-a1fd-ae23c0a468c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:58.991971 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472121e3-a82c-4dce-a1fd-ae23c0a468c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:58.995298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472121e3-a82c-4dce-a1fd-ae23c0a468c7 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:59.094253 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:59.183822 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 428/868] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:41:58 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 14:41:59.194556 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a42ab9-fe3f-4275-a0da-26ba5ab77d80 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:59.407961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a42ab9-fe3f-4275-a0da-26ba5ab77d80 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:59.479762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd969f1-3146-44fd-a26c-81264ee76f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:59.480885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd969f1-3146-44fd-a26c-81264ee76f30 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:59.481373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd969f1-3146-44fd-a26c-81264ee76f30 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:59.481842 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd969f1-3146-44fd-a26c-81264ee76f30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:59.504685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd969f1-3146-44fd-a26c-81264ee76f30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:59.507488 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd969f1-3146-44fd-a26c-81264ee76f30 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:59.518194 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-1fd969f1-3146-44fd-a26c-81264ee76f30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:41:59.535099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:41:59.554679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:41:59.556507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:41:59.556507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:41:59.556507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:41:59.556507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:41:59.556507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 441/869] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:41:59.557032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472121e3-a82c-4dce-a1fd-ae23c0a468c7 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:59.689444 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b976f9f-b6b0-404f-a419-044201e7a67a None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:59.689879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b976f9f-b6b0-404f-a419-044201e7a67a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:59.690342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b976f9f-b6b0-404f-a419-044201e7a67a None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:59.695516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 442/870] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:41:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:41:59.702231 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df48cae-b5cd-4c50-a51d-a7d44601468f None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:59.703425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df48cae-b5cd-4c50-a51d-a7d44601468f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:59.704027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df48cae-b5cd-4c50-a51d-a7d44601468f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:59.736253 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-0df48cae-b5cd-4c50-a51d-a7d44601468f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:41:59.739191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-0df48cae-b5cd-4c50-a51d-a7d44601468f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:41:59.761243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472121e3-a82c-4dce-a1fd-ae23c0a468c7 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:41:59.833868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21bdb4-3240-4c8f-acb5-296859a07c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:41:59.834891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21bdb4-3240-4c8f-acb5-296859a07c69 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:41:59.835433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21bdb4-3240-4c8f-acb5-296859a07c69 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:41:59.836033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21bdb4-3240-4c8f-acb5-296859a07c69 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:41:59.838030 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21bdb4-3240-4c8f-acb5-296859a07c69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:41:59.838313 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21bdb4-3240-4c8f-acb5-296859a07c69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:41:59.840998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21bdb4-3240-4c8f-acb5-296859a07c69 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:41:59.871774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 429/871] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:41:59.875403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21bdb4-3240-4c8f-acb5-296859a07c69 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:41:59.891012 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:41:59.899623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 443/872] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:41:59 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:41:59.937882 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df48cae-b5cd-4c50-a51d-a7d44601468f None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:00.057062 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21bdb4-3240-4c8f-acb5-296859a07c69 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:00.123123 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df48cae-b5cd-4c50-a51d-a7d44601468f None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:00.135672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623de5c-4647-4e7c-a0eb-92257a1b24b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:00.137651 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623de5c-4647-4e7c-a0eb-92257a1b24b8 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:00.138167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623de5c-4647-4e7c-a0eb-92257a1b24b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:00.138655 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623de5c-4647-4e7c-a0eb-92257a1b24b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:00.150521 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623de5c-4647-4e7c-a0eb-92257a1b24b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:00.152290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623de5c-4647-4e7c-a0eb-92257a1b24b8 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-d623de5c-4647-4e7c-a0eb-92257a1b24b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:42:00.167148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:00.168983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:00.170873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:42:00.170873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:42:00.170873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:42:00.170873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:42:00.170873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:42:00.170873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:42:00.170873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:00.170873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:00.174053 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 430/873] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:41:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:00.175566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd4d4-22fd-45d8-a3e9-1f0cf4a2d674 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:00.175854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd4d4-22fd-45d8-a3e9-1f0cf4a2d674 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:00.176133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd4d4-22fd-45d8-a3e9-1f0cf4a2d674 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:00.176600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:00.177750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 431/874] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:00.178359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bd4d4-22fd-45d8-a3e9-1f0cf4a2d674 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:00.198890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b789f-0d16-4088-ba56-a7f0cdaec3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:00.200059 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b789f-0d16-4088-ba56-a7f0cdaec3a7 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:00.200547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b789f-0d16-4088-ba56-a7f0cdaec3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:00.201151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b789f-0d16-4088-ba56-a7f0cdaec3a7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:00.202463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b789f-0d16-4088-ba56-a7f0cdaec3a7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:00.205549 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b789f-0d16-4088-ba56-a7f0cdaec3a7 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-6f8b789f-0d16-4088-ba56-a7f0cdaec3a7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:00.221096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:00.223103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:00.224744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 444/875] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:41:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:00.226296 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f870f75-992c-45fb-9e7c-d01994c76dd9 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:00.226296 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f870f75-992c-45fb-9e7c-d01994c76dd9 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:00.226296 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f870f75-992c-45fb-9e7c-d01994c76dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:00.256260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-3f870f75-992c-45fb-9e7c-d01994c76dd9 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:00.258445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3f870f75-992c-45fb-9e7c-d01994c76dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:00.344184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:00.371633 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 445/876] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:00.371633 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f870f75-992c-45fb-9e7c-d01994c76dd9 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:00.426103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bd4d4-22fd-45d8-a3e9-1f0cf4a2d674 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:00.494496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5d207-9f7e-4962-8cf6-89bf6b81fa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:00.495312 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5d207-9f7e-4962-8cf6-89bf6b81fa1f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:00.495834 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5d207-9f7e-4962-8cf6-89bf6b81fa1f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:00.496289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5d207-9f7e-4962-8cf6-89bf6b81fa1f None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:00.497532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5d207-9f7e-4962-8cf6-89bf6b81fa1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:00.498294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5d207-9f7e-4962-8cf6-89bf6b81fa1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:00.502077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5d207-9f7e-4962-8cf6-89bf6b81fa1f None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:00.529537 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 432/877] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:42:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:00.530798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5d207-9f7e-4962-8cf6-89bf6b81fa1f None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:00.561727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f870f75-992c-45fb-9e7c-d01994c76dd9 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:00.595299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:00.619348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed91779-bfee-4098-b964-533ad6a7c333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:00.620662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed91779-bfee-4098-b964-533ad6a7c333 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:00.621146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed91779-bfee-4098-b964-533ad6a7c333 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:00.621607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed91779-bfee-4098-b964-533ad6a7c333 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:00.623521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed91779-bfee-4098-b964-533ad6a7c333 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:00.624072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed91779-bfee-4098-b964-533ad6a7c333 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:00.627401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed91779-bfee-4098-b964-533ad6a7c333 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:00.636719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 446/878] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:42:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:00.638260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed91779-bfee-4098-b964-533ad6a7c333 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:00.751633 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5d207-9f7e-4962-8cf6-89bf6b81fa1f None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:00.788553 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:00.802342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9812b9-ab8b-485d-a638-a433bcf47033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:00.803014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9812b9-ab8b-485d-a638-a433bcf47033 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:00.803297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9812b9-ab8b-485d-a638-a433bcf47033 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:00.803702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9812b9-ab8b-485d-a638-a433bcf47033 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:00.804363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9812b9-ab8b-485d-a638-a433bcf47033 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:00.805925 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9812b9-ab8b-485d-a638-a433bcf47033 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-6a9812b9-ab8b-485d-a638-a433bcf47033 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:00.817142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:00.819203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:00.820880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 433/879] 10.222.0.94 () {56 vars in 1126 bytes} [Tue Jul 26 14:42:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:00.822494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9812b9-ab8b-485d-a638-a433bcf47033 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:00.832150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed91779-bfee-4098-b964-533ad6a7c333 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:00.902259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09117818-9615-4c5d-bd58-0ecd23f788e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:00.903315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09117818-9615-4c5d-bd58-0ecd23f788e0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:00.903831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09117818-9615-4c5d-bd58-0ecd23f788e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:00.904318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09117818-9615-4c5d-bd58-0ecd23f788e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:00.905853 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09117818-9615-4c5d-bd58-0ecd23f788e0 None admin] RBAC: Adding query filter params (role.id=170d72cc35f34a249c65a819a18638a4, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=5b24d190536a4953930432287dfbfb9b) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:00.907618 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09117818-9615-4c5d-bd58-0ecd23f788e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:00.910713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09117818-9615-4c5d-bd58-0ecd23f788e0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:00.927511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 447/880] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:42:00 2022] GET /identity/v3/role_assignments?user.id=5b24d190536a4953930432287dfbfb9b&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=170d72cc35f34a249c65a819a18638a4 => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:00.927511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09117818-9615-4c5d-bd58-0ecd23f788e0 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:01.009549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9812b9-ab8b-485d-a638-a433bcf47033 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:01.025746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:01.069950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4ca95-a89b-40e4-9fc9-2653966cbf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:01.070993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4ca95-a89b-40e4-9fc9-2653966cbf3d None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:01.071392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4ca95-a89b-40e4-9fc9-2653966cbf3d None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:01.071807 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4ca95-a89b-40e4-9fc9-2653966cbf3d None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:01.072664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4ca95-a89b-40e4-9fc9-2653966cbf3d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:01.073086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4ca95-a89b-40e4-9fc9-2653966cbf3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:01.074694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4ca95-a89b-40e4-9fc9-2653966cbf3d None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:01.086632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 434/881] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:42:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:01.088107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4ca95-a89b-40e4-9fc9-2653966cbf3d None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:01.142569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09117818-9615-4c5d-bd58-0ecd23f788e0 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:01.163308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:01.216071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec512ee-62fe-4c20-a363-38ad3cf977c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:01.216717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec512ee-62fe-4c20-a363-38ad3cf977c8 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:01.217210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec512ee-62fe-4c20-a363-38ad3cf977c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:01.217632 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec512ee-62fe-4c20-a363-38ad3cf977c8 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:01.218347 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec512ee-62fe-4c20-a363-38ad3cf977c8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:01.219295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec512ee-62fe-4c20-a363-38ad3cf977c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:01.221370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec512ee-62fe-4c20-a363-38ad3cf977c8 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:01.228612 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 448/882] 10.222.0.94 () {56 vars in 1122 bytes} [Tue Jul 26 14:42:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:01.235859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec512ee-62fe-4c20-a363-38ad3cf977c8 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:01.241714 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4ca95-a89b-40e4-9fc9-2653966cbf3d None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:01.335371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04792088-b3bf-40c5-91ea-6e9548dc10d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:01.336345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04792088-b3bf-40c5-91ea-6e9548dc10d3 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:01.336802 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04792088-b3bf-40c5-91ea-6e9548dc10d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:01.337230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04792088-b3bf-40c5-91ea-6e9548dc10d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:01.338510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04792088-b3bf-40c5-91ea-6e9548dc10d3 None admin] RBAC: Adding query filter params (interface=public, service_id=4ba048e231c44c3fb492a49c00aead5e, region_id=RegionOne) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:01.339174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04792088-b3bf-40c5-91ea-6e9548dc10d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:01.342651 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04792088-b3bf-40c5-91ea-6e9548dc10d3 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:01.357969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 435/883] 10.222.0.94 () {56 vars in 1251 bytes} [Tue Jul 26 14:42:01 2022] GET /identity/v3/endpoints?service_id=4ba048e231c44c3fb492a49c00aead5e&interface=public®ion_id=RegionOne => generated 194 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:01.360802 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59ea23-6789-4db8-ba05-26e4e57addc4 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:01.361288 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59ea23-6789-4db8-ba05-26e4e57addc4 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:01.361768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59ea23-6789-4db8-ba05-26e4e57addc4 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:01.365877 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 436/884] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:01.369104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59ea23-6789-4db8-ba05-26e4e57addc4 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:01.445333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec512ee-62fe-4c20-a363-38ad3cf977c8 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:01.466035 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:01.501601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25849b9-5124-46b7-a372-e5abe394a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:01.502898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25849b9-5124-46b7-a372-e5abe394a16d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:01.503173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25849b9-5124-46b7-a372-e5abe394a16d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:01.503690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25849b9-5124-46b7-a372-e5abe394a16d None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:01.505289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25849b9-5124-46b7-a372-e5abe394a16d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:01.506364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25849b9-5124-46b7-a372-e5abe394a16d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:01.509122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25849b9-5124-46b7-a372-e5abe394a16d None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:01.525924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 449/885] 10.222.0.94 () {56 vars in 1122 bytes} [Tue Jul 26 14:42:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:01.525924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a07d7-ba9a-488d-bcef-387d8bfa1479 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:01.525924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a07d7-ba9a-488d-bcef-387d8bfa1479 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:01.525924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a07d7-ba9a-488d-bcef-387d8bfa1479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:01.545087 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59ea23-6789-4db8-ba05-26e4e57addc4 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:01.555294 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-e28a07d7-ba9a-488d-bcef-387d8bfa1479 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:01.557472 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e28a07d7-ba9a-488d-bcef-387d8bfa1479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:01.584418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:01.625740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf605c45-fa50-4cc9-8f0b-1fd85d84b7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:01.628368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf605c45-fa50-4cc9-8f0b-1fd85d84b7d3 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:01.629168 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf605c45-fa50-4cc9-8f0b-1fd85d84b7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:01.629838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf605c45-fa50-4cc9-8f0b-1fd85d84b7d3 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:01.631239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf605c45-fa50-4cc9-8f0b-1fd85d84b7d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:01.634128 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf605c45-fa50-4cc9-8f0b-1fd85d84b7d3 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:01.655792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 437/886] 10.222.0.94 () {56 vars in 1087 bytes} [Tue Jul 26 14:42:01 2022] GET /identity/v3/services => generated 1011 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:42:01.700587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 450/887] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:01.725860 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf605c45-fa50-4cc9-8f0b-1fd85d84b7d3 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:01.892726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf605c45-fa50-4cc9-8f0b-1fd85d84b7d3 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:01.967039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad1b99-2e66-42f5-a21a-98f09fca23b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:01.968589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad1b99-2e66-42f5-a21a-98f09fca23b0 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:01.969372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad1b99-2e66-42f5-a21a-98f09fca23b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:01.970069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad1b99-2e66-42f5-a21a-98f09fca23b0 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:01.971700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad1b99-2e66-42f5-a21a-98f09fca23b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:01.974637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad1b99-2e66-42f5-a21a-98f09fca23b0 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:02.082172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 438/888] 10.222.0.94 () {60 vars in 1141 bytes} [Tue Jul 26 14:42:01 2022] POST /identity/v3/services => generated 269 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:42:04.189820 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1e5b2-9bf0-4b7c-8915-1aff111209fb None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:04.190659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1e5b2-9bf0-4b7c-8915-1aff111209fb None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:04.190659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1e5b2-9bf0-4b7c-8915-1aff111209fb None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:04.192190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 451/889] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:04.198668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddcf42-b359-4a67-890d-d8a22d688788 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:04.199688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddcf42-b359-4a67-890d-d8a22d688788 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:04.200306 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddcf42-b359-4a67-890d-d8a22d688788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:04.228266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-58ddcf42-b359-4a67-890d-d8a22d688788 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:04.230399 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-58ddcf42-b359-4a67-890d-d8a22d688788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:04.313995 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:04.390533 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 439/890] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:04.417566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1e5b2-9bf0-4b7c-8915-1aff111209fb None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:04.567103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1e5b2-9bf0-4b7c-8915-1aff111209fb None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:04.579205 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:04.588110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cdf6d-a016-49a2-96ad-96ae855672c5 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:04.588581 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cdf6d-a016-49a2-96ad-96ae855672c5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:04.589120 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cdf6d-a016-49a2-96ad-96ae855672c5 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:04.591750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 440/891] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:42:04.595717 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3c267-f5e7-4c4a-949b-c8003cc24be0 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:04.596546 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3c267-f5e7-4c4a-949b-c8003cc24be0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:04.602679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3c267-f5e7-4c4a-949b-c8003cc24be0 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:04.602679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 441/892] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:04.602679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bfc7f-1e64-4908-bf07-3acb88cec8f1 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:04.603649 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bfc7f-1e64-4908-bf07-3acb88cec8f1 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:04.604435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bfc7f-1e64-4908-bf07-3acb88cec8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:04.615151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8582469-3647-4329-984a-5bfab8689814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:04.615948 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8582469-3647-4329-984a-5bfab8689814 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:04.616273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8582469-3647-4329-984a-5bfab8689814 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:04.616459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8582469-3647-4329-984a-5bfab8689814 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:04.628983 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8582469-3647-4329-984a-5bfab8689814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-b8582469-3647-4329-984a-5bfab8689814 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:42:04.634848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:04.637660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:04.638521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:04.638521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:04.638521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:04.638521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:04.638521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:42:04.638521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:42:04.638521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:42:04.638521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:42:04.638521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:42:04.638521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:42:04.638521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:42:04.638521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:04.639171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-532bfc7f-1e64-4908-bf07-3acb88cec8f1 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:04.639246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-532bfc7f-1e64-4908-bf07-3acb88cec8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:04.640929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 452/893] 10.222.0.94 () {56 vars in 1120 bytes} [Tue Jul 26 14:42:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:04.644307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136695a-25f8-40b2-8434-1e0956a7d8b5 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:04.644827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136695a-25f8-40b2-8434-1e0956a7d8b5 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:04.646651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136695a-25f8-40b2-8434-1e0956a7d8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:04.672328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c136695a-25f8-40b2-8434-1e0956a7d8b5 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:04.673637 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c136695a-25f8-40b2-8434-1e0956a7d8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:04.744376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 442/894] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:04.745713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bfc7f-1e64-4908-bf07-3acb88cec8f1 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:04.783133 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 453/895] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:04.784740 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136695a-25f8-40b2-8434-1e0956a7d8b5 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:04.896397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:04.904046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bfc7f-1e64-4908-bf07-3acb88cec8f1 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:04.950357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:04.956163 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136695a-25f8-40b2-8434-1e0956a7d8b5 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:04.972143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53a2c5-59db-40e4-942a-ceb76673e276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:04.973203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53a2c5-59db-40e4-942a-ceb76673e276 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:04.973651 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53a2c5-59db-40e4-942a-ceb76673e276 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:04.974050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53a2c5-59db-40e4-942a-ceb76673e276 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:04.975309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53a2c5-59db-40e4-942a-ceb76673e276 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:04.975865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53a2c5-59db-40e4-942a-ceb76673e276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:04.978599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53a2c5-59db-40e4-942a-ceb76673e276 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:05.003527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 443/896] 10.222.0.94 () {56 vars in 1115 bytes} [Tue Jul 26 14:42:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:05.003527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53a2c5-59db-40e4-942a-ceb76673e276 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:05.009288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9bd80-c9eb-4c41-a7c0-1fc5ec6a09ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:05.010070 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9bd80-c9eb-4c41-a7c0-1fc5ec6a09ac None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:05.010497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9bd80-c9eb-4c41-a7c0-1fc5ec6a09ac None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:05.010915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9bd80-c9eb-4c41-a7c0-1fc5ec6a09ac None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:05.012115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9bd80-c9eb-4c41-a7c0-1fc5ec6a09ac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:05.014687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9bd80-c9eb-4c41-a7c0-1fc5ec6a09ac None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:05.122911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 454/897] 10.222.0.94 () {60 vars in 1141 bytes} [Tue Jul 26 14:42:04 2022] POST /identity/v3/services => generated 241 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:42:05.125551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9bd80-c9eb-4c41-a7c0-1fc5ec6a09ac None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:05.177155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53a2c5-59db-40e4-942a-ceb76673e276 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:05.267824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3fb4c-33ad-4098-a9e8-50f981b83d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:05.270657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3fb4c-33ad-4098-a9e8-50f981b83d16 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:05.271065 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3fb4c-33ad-4098-a9e8-50f981b83d16 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:05.271413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3fb4c-33ad-4098-a9e8-50f981b83d16 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:05.272376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3fb4c-33ad-4098-a9e8-50f981b83d16 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:05.274404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3fb4c-33ad-4098-a9e8-50f981b83d16 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-86f3fb4c-33ad-4098-a9e8-50f981b83d16 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:05.289294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:05.291257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:05.294125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 444/898] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:42:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:05.295783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c00d8-8ce6-4c1c-bab7-4863f081b798 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:05.295783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c00d8-8ce6-4c1c-bab7-4863f081b798 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:05.295783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c00d8-8ce6-4c1c-bab7-4863f081b798 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:05.305329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 445/899] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:05 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:05.306839 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c00d8-8ce6-4c1c-bab7-4863f081b798 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:05.349754 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:05.354606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9bd80-c9eb-4c41-a7c0-1fc5ec6a09ac None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:05.412274 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16253f48-493d-46e0-a6ee-3776a71ab6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:05.413309 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16253f48-493d-46e0-a6ee-3776a71ab6d6 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:05.413598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16253f48-493d-46e0-a6ee-3776a71ab6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:05.413980 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16253f48-493d-46e0-a6ee-3776a71ab6d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:05.430356 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16253f48-493d-46e0-a6ee-3776a71ab6d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:05.432757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16253f48-493d-46e0-a6ee-3776a71ab6d6 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-16253f48-493d-46e0-a6ee-3776a71ab6d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:42:05.455198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:05.456361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:05.457386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:42:05.457386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:42:05.457386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:05.457386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:05.457386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 455/900] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:05.458523 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b0652-e639-49d3-ada8-6b50e8bf407b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:05.458862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b0652-e639-49d3-ada8-6b50e8bf407b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:05.459170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b0652-e639-49d3-ada8-6b50e8bf407b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:05.484565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-5f4b0652-e639-49d3-ada8-6b50e8bf407b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:05.486631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5f4b0652-e639-49d3-ada8-6b50e8bf407b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:05.492269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:05.506128 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c00d8-8ce6-4c1c-bab7-4863f081b798 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:05.557159 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959c029-919e-42a5-a79d-c438d7cb7d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:05.558257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959c029-919e-42a5-a79d-c438d7cb7d3a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:05.558762 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959c029-919e-42a5-a79d-c438d7cb7d3a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:05.559264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959c029-919e-42a5-a79d-c438d7cb7d3a None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:05.560392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959c029-919e-42a5-a79d-c438d7cb7d3a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:05.560785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959c029-919e-42a5-a79d-c438d7cb7d3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:05.562971 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959c029-919e-42a5-a79d-c438d7cb7d3a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:05.577164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 446/901] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:42:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:05.577164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959c029-919e-42a5-a79d-c438d7cb7d3a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:05.617956 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 456/902] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:05.619601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b0652-e639-49d3-ada8-6b50e8bf407b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:05.730096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:05.757251 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959c029-919e-42a5-a79d-c438d7cb7d3a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:05.787264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b0652-e639-49d3-ada8-6b50e8bf407b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:05.809669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d35080-17d6-4654-a914-4c0a9b80e89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:05.810463 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d35080-17d6-4654-a914-4c0a9b80e89e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:05.810930 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d35080-17d6-4654-a914-4c0a9b80e89e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:05.811256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d35080-17d6-4654-a914-4c0a9b80e89e None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:05.813837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d35080-17d6-4654-a914-4c0a9b80e89e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:05.817641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d35080-17d6-4654-a914-4c0a9b80e89e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:05.820216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d35080-17d6-4654-a914-4c0a9b80e89e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:05.842186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 447/903] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:05.844133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d35080-17d6-4654-a914-4c0a9b80e89e None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:05.850002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8bdee-45c9-4844-a053-2ab31be3674c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:05.852806 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8bdee-45c9-4844-a053-2ab31be3674c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:05.853303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8bdee-45c9-4844-a053-2ab31be3674c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:05.853753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8bdee-45c9-4844-a053-2ab31be3674c None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:05.854937 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8bdee-45c9-4844-a053-2ab31be3674c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:05.855427 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8bdee-45c9-4844-a053-2ab31be3674c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:05.857872 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8bdee-45c9-4844-a053-2ab31be3674c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:05.871543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 457/904] 10.222.0.94 () {56 vars in 1114 bytes} [Tue Jul 26 14:42:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:05.871543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8bdee-45c9-4844-a053-2ab31be3674c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:05.878711 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:06.019693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d35080-17d6-4654-a914-4c0a9b80e89e None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:06.056595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8bdee-45c9-4844-a053-2ab31be3674c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:06.090535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686f3fd-298d-461f-98b4-d97d9c5985fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:06.091857 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686f3fd-298d-461f-98b4-d97d9c5985fc None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:06.092367 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686f3fd-298d-461f-98b4-d97d9c5985fc None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:06.092872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686f3fd-298d-461f-98b4-d97d9c5985fc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:06.094395 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686f3fd-298d-461f-98b4-d97d9c5985fc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:06.097150 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686f3fd-298d-461f-98b4-d97d9c5985fc None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:06.102636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294846fc-4a14-4116-986a-4c51da3f2dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:06.103453 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294846fc-4a14-4116-986a-4c51da3f2dae None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:06.103838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294846fc-4a14-4116-986a-4c51da3f2dae None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:06.104173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294846fc-4a14-4116-986a-4c51da3f2dae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:06.114782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294846fc-4a14-4116-986a-4c51da3f2dae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:06.116318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294846fc-4a14-4116-986a-4c51da3f2dae None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-f686f3fd-298d-461f-98b4-d97d9c5985fc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:06.117561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:06.119629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:42:06.121552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:06.126366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 448/905] 10.222.0.94 () {56 vars in 1132 bytes} [Tue Jul 26 14:42:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:06.128397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686f3fd-298d-461f-98b4-d97d9c5985fc None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-294846fc-4a14-4116-986a-4c51da3f2dae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:42:06.136655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:06.138521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:06.141582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:06.141582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:06.141582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:42:06.141582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:42:06.141582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:42:06.141582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:42:06.141582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:42:06.141582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:42:06.141582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:42:06.141582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:06.141582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 458/906] 10.222.0.94 () {56 vars in 1117 bytes} [Tue Jul 26 14:42:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:06.141582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294846fc-4a14-4116-986a-4c51da3f2dae None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:06.150900 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:06.277737 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:06.322513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686f3fd-298d-461f-98b4-d97d9c5985fc None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:06.334845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294846fc-4a14-4116-986a-4c51da3f2dae None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:06.383680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf9399-e572-42f6-b0a8-300eaf26f194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:06.384305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf9399-e572-42f6-b0a8-300eaf26f194 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:06.384582 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf9399-e572-42f6-b0a8-300eaf26f194 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:06.384855 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf9399-e572-42f6-b0a8-300eaf26f194 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:06.385585 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf9399-e572-42f6-b0a8-300eaf26f194 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:06.386974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf9399-e572-42f6-b0a8-300eaf26f194 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:06.392207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493f04a-177f-4b1c-a299-3a25330ab9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:06.393268 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493f04a-177f-4b1c-a299-3a25330ab9cc None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:06.393757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493f04a-177f-4b1c-a299-3a25330ab9cc None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:06.394320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493f04a-177f-4b1c-a299-3a25330ab9cc None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:06.395631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493f04a-177f-4b1c-a299-3a25330ab9cc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:06.396188 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493f04a-177f-4b1c-a299-3a25330ab9cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:06.398581 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493f04a-177f-4b1c-a299-3a25330ab9cc None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:06.406895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 459/907] 10.222.0.94 () {56 vars in 1126 bytes} [Tue Jul 26 14:42:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:06.408569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493f04a-177f-4b1c-a299-3a25330ab9cc None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:06.531594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 449/908] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 14:42:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:42:06.531594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf9399-e572-42f6-b0a8-300eaf26f194 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:06.616038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493f04a-177f-4b1c-a299-3a25330ab9cc None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:06.649318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:06.692093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:06.726663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48d312-d952-44cf-bc04-cb0aadce1266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:06.727518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48d312-d952-44cf-bc04-cb0aadce1266 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:06.727890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48d312-d952-44cf-bc04-cb0aadce1266 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:06.728151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48d312-d952-44cf-bc04-cb0aadce1266 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:06.728967 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48d312-d952-44cf-bc04-cb0aadce1266 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:06.729273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48d312-d952-44cf-bc04-cb0aadce1266 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:06.730861 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48d312-d952-44cf-bc04-cb0aadce1266 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:06.731346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf9399-e572-42f6-b0a8-300eaf26f194 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:06.759814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 460/909] 10.222.0.94 () {56 vars in 1149 bytes} [Tue Jul 26 14:42:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:06.769226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48d312-d952-44cf-bc04-cb0aadce1266 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:06.816280 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e367c1-6947-4fa7-a8de-4a4659c90852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:06.817028 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e367c1-6947-4fa7-a8de-4a4659c90852 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:06.817487 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e367c1-6947-4fa7-a8de-4a4659c90852 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:06.817773 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e367c1-6947-4fa7-a8de-4a4659c90852 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:06.818563 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e367c1-6947-4fa7-a8de-4a4659c90852 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:06.818876 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e367c1-6947-4fa7-a8de-4a4659c90852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:06.820763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e367c1-6947-4fa7-a8de-4a4659c90852 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:06.831414 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 450/910] 10.222.0.94 () {56 vars in 1126 bytes} [Tue Jul 26 14:42:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:06.838025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e367c1-6947-4fa7-a8de-4a4659c90852 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:06.933891 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48d312-d952-44cf-bc04-cb0aadce1266 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:06.984439 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16058b44-2da0-4d60-8df0-d61450ecca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:06.985072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16058b44-2da0-4d60-8df0-d61450ecca76 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:06.985350 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16058b44-2da0-4d60-8df0-d61450ecca76 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:06.985643 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16058b44-2da0-4d60-8df0-d61450ecca76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:07.072685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:07.083923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e367c1-6947-4fa7-a8de-4a4659c90852 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:07.105009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16058b44-2da0-4d60-8df0-d61450ecca76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:07.110272 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16058b44-2da0-4d60-8df0-d61450ecca76 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:07.139994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:07.162328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ba4d2-62be-426f-b086-9fac3c490ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:07.163675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ba4d2-62be-426f-b086-9fac3c490ba0 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:07.164286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ba4d2-62be-426f-b086-9fac3c490ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:07.164881 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ba4d2-62be-426f-b086-9fac3c490ba0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:07.166712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ba4d2-62be-426f-b086-9fac3c490ba0 None admin] RBAC: Adding query filter params (interface=public, service_id=2a8537f727de4fdeb6d75a03f7a4b9b1, region_id=RegionOne) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:07.167174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ba4d2-62be-426f-b086-9fac3c490ba0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:07.170231 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ba4d2-62be-426f-b086-9fac3c490ba0 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:07.183385 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 451/911] 10.222.0.94 () {56 vars in 1251 bytes} [Tue Jul 26 14:42:06 2022] GET /identity/v3/endpoints?service_id=2a8537f727de4fdeb6d75a03f7a4b9b1&interface=public®ion_id=RegionOne => generated 194 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:07.191775 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ba4d2-62be-426f-b086-9fac3c490ba0 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:07.344456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 461/912] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:42:06 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:42:07.350040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16058b44-2da0-4d60-8df0-d61450ecca76 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:07.398757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ba4d2-62be-426f-b086-9fac3c490ba0 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:07.445238 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5618b6-f25a-469d-bd08-e5b71856d32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:07.445931 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5618b6-f25a-469d-bd08-e5b71856d32a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:07.446347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5618b6-f25a-469d-bd08-e5b71856d32a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:07.446711 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5618b6-f25a-469d-bd08-e5b71856d32a None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:07.447453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5618b6-f25a-469d-bd08-e5b71856d32a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:07.449375 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5618b6-f25a-469d-bd08-e5b71856d32a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:07.459210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 452/913] 10.222.0.94 () {56 vars in 1087 bytes} [Tue Jul 26 14:42:07 2022] GET /identity/v3/services => generated 1497 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:42:07.534160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:07.579631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16058b44-2da0-4d60-8df0-d61450ecca76 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:07.659027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ec035-e6b0-474d-be98-5de14ddb98be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:07.660297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ec035-e6b0-474d-be98-5de14ddb98be None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:07.660771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ec035-e6b0-474d-be98-5de14ddb98be None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:07.661244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ec035-e6b0-474d-be98-5de14ddb98be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:07.681998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ec035-e6b0-474d-be98-5de14ddb98be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:07.683500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ec035-e6b0-474d-be98-5de14ddb98be None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-152ec035-e6b0-474d-be98-5de14ddb98be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:42:07.706349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:07.708142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:07.709864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:42:07.709864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:42:07.709864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:07.709864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:07.709864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 462/914] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:07.716554 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5618b6-f25a-469d-bd08-e5b71856d32a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:07.748859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ccb77-258b-4f48-b66e-c15cd85a04bc None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:07.749117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ccb77-258b-4f48-b66e-c15cd85a04bc None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:07.749485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ccb77-258b-4f48-b66e-c15cd85a04bc None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:07.752165 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 463/915] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:42:07.760036 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f428f7-c659-4b2e-83d7-c3824ab9adbc None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:07.760715 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f428f7-c659-4b2e-83d7-c3824ab9adbc None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:07.761363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f428f7-c659-4b2e-83d7-c3824ab9adbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:07.798423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c6f428f7-c659-4b2e-83d7-c3824ab9adbc None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:07.800506 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c6f428f7-c659-4b2e-83d7-c3824ab9adbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:07.804071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:07.878028 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5618b6-f25a-469d-bd08-e5b71856d32a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:07.946290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 464/916] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:07.955008 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83782713-1d61-43c2-bc83-8b68aa4e69b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:07.955955 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83782713-1d61-43c2-bc83-8b68aa4e69b5 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:07.956367 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83782713-1d61-43c2-bc83-8b68aa4e69b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:07.956766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83782713-1d61-43c2-bc83-8b68aa4e69b5 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:07.957861 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83782713-1d61-43c2-bc83-8b68aa4e69b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:07.958416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83782713-1d61-43c2-bc83-8b68aa4e69b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:07.961026 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83782713-1d61-43c2-bc83-8b68aa4e69b5 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:07.970787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f428f7-c659-4b2e-83d7-c3824ab9adbc None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:07.985838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 453/917] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:07.994914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83782713-1d61-43c2-bc83-8b68aa4e69b5 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:08.113165 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:08.184470 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f428f7-c659-4b2e-83d7-c3824ab9adbc None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:08.204002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83782713-1d61-43c2-bc83-8b68aa4e69b5 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:08.252075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:08.253106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88c7fe-535e-44ba-a529-9df90cd6eba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:08.254101 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88c7fe-535e-44ba-a529-9df90cd6eba7 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:08.254539 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88c7fe-535e-44ba-a529-9df90cd6eba7 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:08.255553 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88c7fe-535e-44ba-a529-9df90cd6eba7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:08.256263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88c7fe-535e-44ba-a529-9df90cd6eba7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:08.258954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88c7fe-535e-44ba-a529-9df90cd6eba7 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-1c88c7fe-535e-44ba-a529-9df90cd6eba7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:08.272758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:08.275324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:08.277847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 465/918] 10.222.0.94 () {56 vars in 1126 bytes} [Tue Jul 26 14:42:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:08.281760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88c7fe-535e-44ba-a529-9df90cd6eba7 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:08.294417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b72ff-3165-459a-be65-6457d582891a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:08.296832 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b72ff-3165-459a-be65-6457d582891a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:08.297471 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b72ff-3165-459a-be65-6457d582891a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:08.298067 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b72ff-3165-459a-be65-6457d582891a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:08.313723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b72ff-3165-459a-be65-6457d582891a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:08.317618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b72ff-3165-459a-be65-6457d582891a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-bc6b72ff-3165-459a-be65-6457d582891a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:42:08.346441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:08.348493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:08.350601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:42:08.350601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:42:08.350601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:42:08.350601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:42:08.350601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:42:08.350601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:42:08.350601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:08.350601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:08.359629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 454/919] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:42:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:08.364335 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b72ff-3165-459a-be65-6457d582891a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:08.517818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88c7fe-535e-44ba-a529-9df90cd6eba7 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:08.554137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b72ff-3165-459a-be65-6457d582891a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:08.576622 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:08.600543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f556913-eab4-4ff7-8dbd-5bc186c2d6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:08.602235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f556913-eab4-4ff7-8dbd-5bc186c2d6f9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:08.603533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f556913-eab4-4ff7-8dbd-5bc186c2d6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:08.604325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f556913-eab4-4ff7-8dbd-5bc186c2d6f9 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:08.606393 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f556913-eab4-4ff7-8dbd-5bc186c2d6f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:08.608151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f556913-eab4-4ff7-8dbd-5bc186c2d6f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:08.612148 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f556913-eab4-4ff7-8dbd-5bc186c2d6f9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:08.622123 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 466/920] 10.222.0.94 () {56 vars in 1122 bytes} [Tue Jul 26 14:42:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:08.628781 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f556913-eab4-4ff7-8dbd-5bc186c2d6f9 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:08.638476 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2b0e2-ae03-4fcc-bccc-aa31e44671ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:08.639978 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2b0e2-ae03-4fcc-bccc-aa31e44671ca None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:08.642953 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2b0e2-ae03-4fcc-bccc-aa31e44671ca None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:08.643703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2b0e2-ae03-4fcc-bccc-aa31e44671ca None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:08.646360 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2b0e2-ae03-4fcc-bccc-aa31e44671ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:08.647806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2b0e2-ae03-4fcc-bccc-aa31e44671ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:08.649563 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2b0e2-ae03-4fcc-bccc-aa31e44671ca None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:08.674683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 455/921] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:42:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:08.684524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2b0e2-ae03-4fcc-bccc-aa31e44671ca None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:08.740812 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:08.861723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f556913-eab4-4ff7-8dbd-5bc186c2d6f9 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:08.887910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2b0e2-ae03-4fcc-bccc-aa31e44671ca None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:08.931860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6ece0-e63f-4bca-a3bb-470d5b86e1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:08.933405 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6ece0-e63f-4bca-a3bb-470d5b86e1b9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:08.934171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6ece0-e63f-4bca-a3bb-470d5b86e1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:08.934935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6ece0-e63f-4bca-a3bb-470d5b86e1b9 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:08.936612 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6ece0-e63f-4bca-a3bb-470d5b86e1b9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:08.937699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6ece0-e63f-4bca-a3bb-470d5b86e1b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:08.940605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6ece0-e63f-4bca-a3bb-470d5b86e1b9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:08.957375 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 467/922] 10.222.0.94 () {56 vars in 1122 bytes} [Tue Jul 26 14:42:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:08.959523 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6ece0-e63f-4bca-a3bb-470d5b86e1b9 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:08.964264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330930a-8fd4-4ba5-80d1-8d3dd7bfa570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:08.966941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330930a-8fd4-4ba5-80d1-8d3dd7bfa570 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:08.967805 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330930a-8fd4-4ba5-80d1-8d3dd7bfa570 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:08.969122 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330930a-8fd4-4ba5-80d1-8d3dd7bfa570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:08.970842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330930a-8fd4-4ba5-80d1-8d3dd7bfa570 None admin] RBAC: Adding query filter params (role.id=57e2f79cce1a4685848082d34f1eb041, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=5b24d190536a4953930432287dfbfb9b) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:08.971764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330930a-8fd4-4ba5-80d1-8d3dd7bfa570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:08.974650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330930a-8fd4-4ba5-80d1-8d3dd7bfa570 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:08.995796 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 456/923] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:42:08 2022] GET /identity/v3/role_assignments?user.id=5b24d190536a4953930432287dfbfb9b&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=57e2f79cce1a4685848082d34f1eb041 => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:09.085802 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:09.172601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4813d6-6c1a-4288-94b9-3b1c9e73a10d None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:09.172852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4813d6-6c1a-4288-94b9-3b1c9e73a10d None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:09.173177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4813d6-6c1a-4288-94b9-3b1c9e73a10d None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:09.174883 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 457/924] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:42:09.180573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6ece0-e63f-4bca-a3bb-470d5b86e1b9 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:09.180988 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7503e-f79e-4fd3-9abd-53dbbc8af95f None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:09.181143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7503e-f79e-4fd3-9abd-53dbbc8af95f None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:09.181435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7503e-f79e-4fd3-9abd-53dbbc8af95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:09.217763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-aee7503e-f79e-4fd3-9abd-53dbbc8af95f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:09.221388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-aee7503e-f79e-4fd3-9abd-53dbbc8af95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:09.237658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9372e-deb0-4725-9631-8904242e3b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:09.238894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9372e-deb0-4725-9631-8904242e3b12 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:09.239366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9372e-deb0-4725-9631-8904242e3b12 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:09.239960 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9372e-deb0-4725-9631-8904242e3b12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:09.241302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9372e-deb0-4725-9631-8904242e3b12 None admin] RBAC: Adding query filter params (interface=public, service_id=0ee57b62de0d492e85bb6677f367f84c, region_id=RegionOne) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:09.242412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9372e-deb0-4725-9631-8904242e3b12 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:09.244386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9372e-deb0-4725-9631-8904242e3b12 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:09.267506 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 468/925] 10.222.0.94 () {56 vars in 1251 bytes} [Tue Jul 26 14:42:08 2022] GET /identity/v3/endpoints?service_id=0ee57b62de0d492e85bb6677f367f84c&interface=public®ion_id=RegionOne => generated 194 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:09.277580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9372e-deb0-4725-9631-8904242e3b12 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:09.403648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 458/926] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:09.443138 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7503e-f79e-4fd3-9abd-53dbbc8af95f None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:09.490257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9372e-deb0-4725-9631-8904242e3b12 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:09.503350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:09.527132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:09.558129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2503bd-5f24-45a1-8eda-eab5381aa73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:09.559774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2503bd-5f24-45a1-8eda-eab5381aa73f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:09.560360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2503bd-5f24-45a1-8eda-eab5381aa73f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:09.560835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2503bd-5f24-45a1-8eda-eab5381aa73f None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:09.561702 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2503bd-5f24-45a1-8eda-eab5381aa73f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:09.564273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2503bd-5f24-45a1-8eda-eab5381aa73f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:09.583510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 469/927] 10.222.0.94 () {56 vars in 1087 bytes} [Tue Jul 26 14:42:09 2022] GET /identity/v3/services => generated 1497 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:42:09.658314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7503e-f79e-4fd3-9abd-53dbbc8af95f None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:09.712547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ca370-fbd3-4472-b73e-1403768b91e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:09.713843 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ca370-fbd3-4472-b73e-1403768b91e9 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:09.714565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ca370-fbd3-4472-b73e-1403768b91e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:09.715081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ca370-fbd3-4472-b73e-1403768b91e9 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:09.716136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ca370-fbd3-4472-b73e-1403768b91e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:09.717988 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ca370-fbd3-4472-b73e-1403768b91e9 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:09.722256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: WARNING keystone.server.flask.common [None req-3c8ca370-fbd3-4472-b73e-1403768b91e9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 26 14:42:09.781913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.sql.core [None req-3c8ca370-fbd3-4472-b73e-1403768b91e9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 14:42:09.781913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 14:42:09.781913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [parameters: {'id': 'e878c8ac079e4cf5aeaeab7bda912d1d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 14:42:09.781913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92209) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 14:42:09.798140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-3c8ca370-fbd3-4472-b73e-1403768b91e9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e878c8ac079e4cf5aeaeab7bda912d1d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e878c8ac079e4cf5aeaeab7bda912d1d. Jul 26 14:42:09.798140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:09.798140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 14:42:09.798140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 14:42:09.798140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 14:42:09.798140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 14:42:09.798140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 14:42:09.798140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 14:42:09.798140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 14:42:09.798140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application conn.query(q) Jul 26 14:42:09.798140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 14:42:09.798140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 14:42:09.798140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application result.read() Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 26 14:42:09.799245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application next(self.gen) Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application yield resource Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application next(self.gen) Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise self.value Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application session.commit() Jul 26 14:42:09.800722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self.session.flush() Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application compat.raise_( Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 14:42:09.801790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 14:42:09.802867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application conn.query(q) Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application result.read() Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 14:42:09.805116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [parameters: {'id': 'e878c8ac079e4cf5aeaeab7bda912d1d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:09.806542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:09.807673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:09.811401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:09.811401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 26 14:42:09.811401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 26 14:42:09.811401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:09.811401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:09.811401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 26 14:42:09.811401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 26 14:42:09.811401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e878c8ac079e4cf5aeaeab7bda912d1d. Jul 26 14:42:09.811401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:09.813623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 459/928] 10.222.0.94 () {60 vars in 1140 bytes} [Tue Jul 26 14:42:09 2022] POST /identity/v3/projects => generated 262 bytes in 372 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:09.820491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2503bd-5f24-45a1-8eda-eab5381aa73f None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:10.004044 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2503bd-5f24-45a1-8eda-eab5381aa73f None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:10.070806 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01d48c-3b20-4dfc-824a-01beb8772f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:10.071971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01d48c-3b20-4dfc-824a-01beb8772f4f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:10.072461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01d48c-3b20-4dfc-824a-01beb8772f4f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:10.072953 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01d48c-3b20-4dfc-824a-01beb8772f4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:10.086551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01d48c-3b20-4dfc-824a-01beb8772f4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:10.089260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01d48c-3b20-4dfc-824a-01beb8772f4f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-7f01d48c-3b20-4dfc-824a-01beb8772f4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:42:10.115369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:10.117719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:10.119918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:42:10.119918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:42:10.119918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:42:10.119918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:42:10.119918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:42:10.119918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:42:10.119918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:10.119918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:10.119918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 470/929] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:10.128293 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ca370-fbd3-4472-b73e-1403768b91e9 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:10.178766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a52dd-4a8e-4696-a688-c0bfbdc1531d None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:10.179261 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a52dd-4a8e-4696-a688-c0bfbdc1531d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:10.179801 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a52dd-4a8e-4696-a688-c0bfbdc1531d None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:10.182534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 471/930] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:10.192435 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dca22f-df3f-45ee-abbe-fc58cb1e36ed None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:10.193072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dca22f-df3f-45ee-abbe-fc58cb1e36ed None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:10.193554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dca22f-df3f-45ee-abbe-fc58cb1e36ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:10.229226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-41dca22f-df3f-45ee-abbe-fc58cb1e36ed None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:10.231370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-41dca22f-df3f-45ee-abbe-fc58cb1e36ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:10.303742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:10.328814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:10.332028 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ca370-fbd3-4472-b73e-1403768b91e9 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:10.385793 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdcea7-6ca9-4340-8677-817ce7b17ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:10.386469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdcea7-6ca9-4340-8677-817ce7b17ee4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:10.386757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdcea7-6ca9-4340-8677-817ce7b17ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:10.387129 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdcea7-6ca9-4340-8677-817ce7b17ee4 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:10.387967 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 472/931] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:42:10.388087 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdcea7-6ca9-4340-8677-817ce7b17ee4 None admin] RBAC: Adding query filter params (name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:10.388352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdcea7-6ca9-4340-8677-817ce7b17ee4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:10.390115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdcea7-6ca9-4340-8677-817ce7b17ee4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:10.416247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 460/932] 10.222.0.94 () {56 vars in 1112 bytes} [Tue Jul 26 14:42:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:10.426113 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dca22f-df3f-45ee-abbe-fc58cb1e36ed None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:10.562786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dca22f-df3f-45ee-abbe-fc58cb1e36ed None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:10.612481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f5029-822e-4e14-a28b-bd1cc30848b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:10.613606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f5029-822e-4e14-a28b-bd1cc30848b8 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:10.614075 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f5029-822e-4e14-a28b-bd1cc30848b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:10.614495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f5029-822e-4e14-a28b-bd1cc30848b8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:10.615610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f5029-822e-4e14-a28b-bd1cc30848b8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:10.617938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f5029-822e-4e14-a28b-bd1cc30848b8 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-107f5029-822e-4e14-a28b-bd1cc30848b8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:10.634367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:10.635680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 14:42:10.636826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:10.642480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 473/933] 10.222.0.94 () {56 vars in 1132 bytes} [Tue Jul 26 14:42:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:10.644708 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdcea7-6ca9-4340-8677-817ce7b17ee4 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:10.863705 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdcea7-6ca9-4340-8677-817ce7b17ee4 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:10.935093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b766855-182e-46bd-a9f7-b0907dfb6a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:10.936208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b766855-182e-46bd-a9f7-b0907dfb6a98 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:10.936671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b766855-182e-46bd-a9f7-b0907dfb6a98 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:10.937641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b766855-182e-46bd-a9f7-b0907dfb6a98 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:10.938840 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b766855-182e-46bd-a9f7-b0907dfb6a98 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:10.939361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b766855-182e-46bd-a9f7-b0907dfb6a98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:10.941741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b766855-182e-46bd-a9f7-b0907dfb6a98 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:10.951911 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 461/934] 10.222.0.94 () {56 vars in 1126 bytes} [Tue Jul 26 14:42:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:10.958862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f5029-822e-4e14-a28b-bd1cc30848b8 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:11.020447 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:11.103977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f5029-822e-4e14-a28b-bd1cc30848b8 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:11.150804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f754b-1bbb-470d-bf98-c3b7c08bebb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:11.151400 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f754b-1bbb-470d-bf98-c3b7c08bebb0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:11.151724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f754b-1bbb-470d-bf98-c3b7c08bebb0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:11.152043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f754b-1bbb-470d-bf98-c3b7c08bebb0 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:11.152872 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f754b-1bbb-470d-bf98-c3b7c08bebb0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:11.153265 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f754b-1bbb-470d-bf98-c3b7c08bebb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:11.154849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f754b-1bbb-470d-bf98-c3b7c08bebb0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:11.166577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 474/935] 10.222.0.94 () {56 vars in 1126 bytes} [Tue Jul 26 14:42:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:11.168133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b766855-182e-46bd-a9f7-b0907dfb6a98 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:11.179889 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:11.344864 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b766855-182e-46bd-a9f7-b0907dfb6a98 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:11.393410 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a56e8-0198-4a3e-871c-719081f2f804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:11.394480 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a56e8-0198-4a3e-871c-719081f2f804 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:11.394973 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a56e8-0198-4a3e-871c-719081f2f804 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:11.395484 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a56e8-0198-4a3e-871c-719081f2f804 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:11.396857 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a56e8-0198-4a3e-871c-719081f2f804 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:11.399504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a56e8-0198-4a3e-871c-719081f2f804 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:11.497054 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 462/936] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 14:42:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:42:11.769342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3ee88-850b-4174-8cc3-119ebdca4c43 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:11.769971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3ee88-850b-4174-8cc3-119ebdca4c43 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:11.770558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3ee88-850b-4174-8cc3-119ebdca4c43 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:11.773688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 475/937] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:11.783936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60329b3-91f1-4756-8400-c998c7357bbd None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:11.784922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60329b3-91f1-4756-8400-c998c7357bbd None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:11.785514 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60329b3-91f1-4756-8400-c998c7357bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:11.820076 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-b60329b3-91f1-4756-8400-c998c7357bbd None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:11.823029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b60329b3-91f1-4756-8400-c998c7357bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:11.940575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:11.971422 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 463/938] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:12.013055 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3ee88-850b-4174-8cc3-119ebdca4c43 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:12.190477 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3ee88-850b-4174-8cc3-119ebdca4c43 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:12.192389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:12.236714 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac942f-67eb-4043-b39e-95e9c6c47af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:12.237476 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac942f-67eb-4043-b39e-95e9c6c47af6 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:12.237757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac942f-67eb-4043-b39e-95e9c6c47af6 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:12.238036 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac942f-67eb-4043-b39e-95e9c6c47af6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:12.254352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac942f-67eb-4043-b39e-95e9c6c47af6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-7eac942f-67eb-4043-b39e-95e9c6c47af6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:42:12.259993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:12.261170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:12.262170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:12.262170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:12.262170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:12.262170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:12.262170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:42:12.262170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:42:12.262170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:42:12.262170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:42:12.262170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:42:12.262170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:42:12.262170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:42:12.262170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:12.262170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 476/939] 10.222.0.94 () {56 vars in 1120 bytes} [Tue Jul 26 14:42:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:12.270953 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60329b3-91f1-4756-8400-c998c7357bbd None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:12.404657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60329b3-91f1-4756-8400-c998c7357bbd None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:12.449208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e139729-3fc7-4693-89ce-0b3b8d964b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:12.449771 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e139729-3fc7-4693-89ce-0b3b8d964b51 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:12.450058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e139729-3fc7-4693-89ce-0b3b8d964b51 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:12.450408 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e139729-3fc7-4693-89ce-0b3b8d964b51 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:12.451281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e139729-3fc7-4693-89ce-0b3b8d964b51 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:12.451632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e139729-3fc7-4693-89ce-0b3b8d964b51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:12.453356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e139729-3fc7-4693-89ce-0b3b8d964b51 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:12.464457 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 464/940] 10.222.0.94 () {56 vars in 1115 bytes} [Tue Jul 26 14:42:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:12.471290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac942f-67eb-4043-b39e-95e9c6c47af6 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:12.600265 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac942f-67eb-4043-b39e-95e9c6c47af6 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:12.646531 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c18fa-728f-48ac-a111-2bdd81e59d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:12.647410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c18fa-728f-48ac-a111-2bdd81e59d47 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:12.647808 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c18fa-728f-48ac-a111-2bdd81e59d47 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:12.648264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c18fa-728f-48ac-a111-2bdd81e59d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:12.661793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c18fa-728f-48ac-a111-2bdd81e59d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:12.663291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c18fa-728f-48ac-a111-2bdd81e59d47 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-265c18fa-728f-48ac-a111-2bdd81e59d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:42:12.680320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:12.683943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:12.690001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:42:12.690001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:42:12.690001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:12.690001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:12.690001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 477/941] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:12.691075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e139729-3fc7-4693-89ce-0b3b8d964b51 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:12.790758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:12.874887 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e139729-3fc7-4693-89ce-0b3b8d964b51 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:12.921882 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d68b26-3295-46ad-9723-32181d00990f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:12.922549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d68b26-3295-46ad-9723-32181d00990f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:12.922946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d68b26-3295-46ad-9723-32181d00990f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:12.923242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d68b26-3295-46ad-9723-32181d00990f None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:12.924229 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d68b26-3295-46ad-9723-32181d00990f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:12.924642 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d68b26-3295-46ad-9723-32181d00990f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:12.926124 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d68b26-3295-46ad-9723-32181d00990f None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:12.929018 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74c378-fdac-40bf-b698-d027bf1e6711 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:12.929812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74c378-fdac-40bf-b698-d027bf1e6711 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:12.929812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74c378-fdac-40bf-b698-d027bf1e6711 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:12.932091 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 478/942] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:12.938706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcc5d2-6c99-42b3-8a7e-d175bda6e704 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:12.938706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcc5d2-6c99-42b3-8a7e-d175bda6e704 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:12.939115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcc5d2-6c99-42b3-8a7e-d175bda6e704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:12.967328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 465/943] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:12.969158 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d68b26-3295-46ad-9723-32181d00990f None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:12.973510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-42dcc5d2-6c99-42b3-8a7e-d175bda6e704 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:12.977461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-42dcc5d2-6c99-42b3-8a7e-d175bda6e704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:13.020227 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:13.116305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 479/944] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:42:13.116861 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d68b26-3295-46ad-9723-32181d00990f None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:13.136430 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:13.144276 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcc5d2-6c99-42b3-8a7e-d175bda6e704 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:13.176440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983f8ee-ac6e-44d7-83a0-879087d979a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:13.177768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983f8ee-ac6e-44d7-83a0-879087d979a4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:13.178244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983f8ee-ac6e-44d7-83a0-879087d979a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:13.179039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983f8ee-ac6e-44d7-83a0-879087d979a4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:13.192888 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983f8ee-ac6e-44d7-83a0-879087d979a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:13.195144 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983f8ee-ac6e-44d7-83a0-879087d979a4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-9983f8ee-ac6e-44d7-83a0-879087d979a4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:42:13.215411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:13.216459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:13.217524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:13.217524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:13.217524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:42:13.217524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:42:13.217524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:42:13.217524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:42:13.217524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:42:13.217524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:42:13.217524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:42:13.217524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:13.227958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 466/945] 10.222.0.94 () {56 vars in 1117 bytes} [Tue Jul 26 14:42:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:13.230735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983f8ee-ac6e-44d7-83a0-879087d979a4 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:13.326457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcc5d2-6c99-42b3-8a7e-d175bda6e704 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:13.381818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3bc3f-0ea7-40f7-8b08-605bbff56fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:13.382380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3bc3f-0ea7-40f7-8b08-605bbff56fa2 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:13.382659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3bc3f-0ea7-40f7-8b08-605bbff56fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:13.382931 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3bc3f-0ea7-40f7-8b08-605bbff56fa2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:13.383658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3bc3f-0ea7-40f7-8b08-605bbff56fa2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:13.385131 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3bc3f-0ea7-40f7-8b08-605bbff56fa2 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:13.393510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983f8ee-ac6e-44d7-83a0-879087d979a4 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-30a3bc3f-0ea7-40f7-8b08-605bbff56fa2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:13.395778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:13.397391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:13.399030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 480/946] 10.222.0.94 () {56 vars in 1126 bytes} [Tue Jul 26 14:42:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:13.411754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537ccd2-7fd1-4731-af36-59fa1d804867 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:13.411754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537ccd2-7fd1-4731-af36-59fa1d804867 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:13.411754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537ccd2-7fd1-4731-af36-59fa1d804867 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:13.411754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 481/947] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:13.411754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537ccd2-7fd1-4731-af36-59fa1d804867 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:13.424011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:13.463372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7af78-0dc3-485e-91e8-c9b62cea68d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:13.464645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7af78-0dc3-485e-91e8-c9b62cea68d4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:13.465048 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7af78-0dc3-485e-91e8-c9b62cea68d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:13.465328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7af78-0dc3-485e-91e8-c9b62cea68d4 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:13.466182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7af78-0dc3-485e-91e8-c9b62cea68d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:13.466493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7af78-0dc3-485e-91e8-c9b62cea68d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:13.468986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7af78-0dc3-485e-91e8-c9b62cea68d4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:13.494803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 467/948] 10.222.0.94 () {56 vars in 1149 bytes} [Tue Jul 26 14:42:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:13.496553 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae7297-0881-464c-8e7a-2038b2bf84b0 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:13.496943 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae7297-0881-464c-8e7a-2038b2bf84b0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:13.497218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae7297-0881-464c-8e7a-2038b2bf84b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:13.527047 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-0cae7297-0881-464c-8e7a-2038b2bf84b0 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:13.529795 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0cae7297-0881-464c-8e7a-2038b2bf84b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:13.574643 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:13.629991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537ccd2-7fd1-4731-af36-59fa1d804867 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:13.674559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 468/949] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:13.674559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae7297-0881-464c-8e7a-2038b2bf84b0 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:13.705176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d37df-6dab-4913-bc9c-6585371fa8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:13.706243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d37df-6dab-4913-bc9c-6585371fa8f6 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:13.706967 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d37df-6dab-4913-bc9c-6585371fa8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:13.707700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d37df-6dab-4913-bc9c-6585371fa8f6 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:13.709335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d37df-6dab-4913-bc9c-6585371fa8f6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:13.709805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d37df-6dab-4913-bc9c-6585371fa8f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:13.712466 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d37df-6dab-4913-bc9c-6585371fa8f6 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:13.722583 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 482/950] 10.222.0.94 () {56 vars in 1122 bytes} [Tue Jul 26 14:42:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:13.724026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d37df-6dab-4913-bc9c-6585371fa8f6 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:13.873180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:13.886856 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae7297-0881-464c-8e7a-2038b2bf84b0 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:13.918192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d37df-6dab-4913-bc9c-6585371fa8f6 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:13.964286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab01f6-3626-4f57-803d-1df759d39ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:13.966105 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab01f6-3626-4f57-803d-1df759d39ba6 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:13.966797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab01f6-3626-4f57-803d-1df759d39ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:13.966972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf0250-11bf-45d1-b116-f183dc8466e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:13.967888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf0250-11bf-45d1-b116-f183dc8466e6 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:13.968367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf0250-11bf-45d1-b116-f183dc8466e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:13.968530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab01f6-3626-4f57-803d-1df759d39ba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:13.969111 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf0250-11bf-45d1-b116-f183dc8466e6 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:13.970039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf0250-11bf-45d1-b116-f183dc8466e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:13.972507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf0250-11bf-45d1-b116-f183dc8466e6 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:13.979423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: WARNING keystone.server.flask.common [None req-65cf0250-11bf-45d1-b116-f183dc8466e6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 26 14:42:13.988732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab01f6-3626-4f57-803d-1df759d39ba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:13.992421 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab01f6-3626-4f57-803d-1df759d39ba6 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-9aab01f6-3626-4f57-803d-1df759d39ba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:42:14.017740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:14.020088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:14.022600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:42:14.022600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:42:14.022600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:14.022600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:14.022600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 469/951] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:14.025436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab01f6-3626-4f57-803d-1df759d39ba6 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:14.052091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:14.063769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.sql.core [None req-65cf0250-11bf-45d1-b116-f183dc8466e6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 14:42:14.063769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 14:42:14.063769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [parameters: {'user_id': '9d9df02295d642b7b919b989eb0c1e81', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 14:42:14.063769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92208) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-65cf0250-11bf-45d1-b116-f183dc8466e6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application conn.query(q) Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application result.read() Jul 26 14:42:14.087520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 14:42:14.089295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application next(self.gen) Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application yield resource Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application next(self.gen) Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise self.value Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application session.commit() Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 14:42:14.091232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self.session.flush() Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application compat.raise_( Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 14:42:14.093174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 14:42:14.094918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application conn.query(q) Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application result.read() Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 14:42:14.099432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [parameters: {'user_id': '9d9df02295d642b7b919b989eb0c1e81', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:14.102409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 26 14:42:14.104562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 26 14:42:14.106604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 26 14:42:14.106604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 26 14:42:14.106604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 14:42:14.106604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 14:42:14.106604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 14:42:14.106604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:14.106604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 483/952] 10.222.0.94 () {60 vars in 1131 bytes} [Tue Jul 26 14:42:13 2022] POST /identity/v3/users => generated 160 bytes in 375 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:14.106604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf0250-11bf-45d1-b116-f183dc8466e6 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:14.222434 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab01f6-3626-4f57-803d-1df759d39ba6 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:14.292577 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0d70b-e9bd-4dbf-8347-e63ee2c90762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:14.293375 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0d70b-e9bd-4dbf-8347-e63ee2c90762 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:14.293822 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0d70b-e9bd-4dbf-8347-e63ee2c90762 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:14.294271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0d70b-e9bd-4dbf-8347-e63ee2c90762 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:14.295486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0d70b-e9bd-4dbf-8347-e63ee2c90762 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:14.296000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0d70b-e9bd-4dbf-8347-e63ee2c90762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:14.298308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0d70b-e9bd-4dbf-8347-e63ee2c90762 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:14.315724 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 470/953] 10.222.0.94 () {56 vars in 1122 bytes} [Tue Jul 26 14:42:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:14.316874 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0d70b-e9bd-4dbf-8347-e63ee2c90762 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:14.321897 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf0250-11bf-45d1-b116-f183dc8466e6 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:14.380738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e0b5a-c4e6-42d4-b4ae-dbb17f2684eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:14.381577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0b5a-c4e6-42d4-b4ae-dbb17f2684eb None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:14.382046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0b5a-c4e6-42d4-b4ae-dbb17f2684eb None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:14.382344 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e0b5a-c4e6-42d4-b4ae-dbb17f2684eb None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:14.383129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e0b5a-c4e6-42d4-b4ae-dbb17f2684eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:14.383573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e0b5a-c4e6-42d4-b4ae-dbb17f2684eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:14.385416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e0b5a-c4e6-42d4-b4ae-dbb17f2684eb None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:14.388031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:14.406110 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 484/954] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:14.407030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e0b5a-c4e6-42d4-b4ae-dbb17f2684eb None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:14.444564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:14.498817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0d70b-e9bd-4dbf-8347-e63ee2c90762 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:14.532428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62f5d9-8b1d-414f-ac91-c85ea7322cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:14.533117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62f5d9-8b1d-414f-ac91-c85ea7322cf3 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:14.533940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62f5d9-8b1d-414f-ac91-c85ea7322cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:14.534587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62f5d9-8b1d-414f-ac91-c85ea7322cf3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:14.545416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62f5d9-8b1d-414f-ac91-c85ea7322cf3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:14.547838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62f5d9-8b1d-414f-ac91-c85ea7322cf3 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-6f62f5d9-8b1d-414f-ac91-c85ea7322cf3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:42:14.563083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:14.565289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:14.567289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:14.567289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:14.567289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:42:14.567289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:42:14.567289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:42:14.567289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:42:14.567289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:42:14.567289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:42:14.567289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 14:42:14.567289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:14.567289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 471/955] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:42:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:14.567289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bb3bd-d147-4ae3-bd63-2491cc2266e4 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:14.567289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bb3bd-d147-4ae3-bd63-2491cc2266e4 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:14.568814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bb3bd-d147-4ae3-bd63-2491cc2266e4 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:14.569525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 472/956] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:14.570683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0bb3bd-d147-4ae3-bd63-2491cc2266e4 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:14.577751 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e0b5a-c4e6-42d4-b4ae-dbb17f2684eb None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:14.649299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe0e5b-05fc-49bc-950c-f81f7213215b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:14.650366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe0e5b-05fc-49bc-950c-f81f7213215b None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:14.650865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe0e5b-05fc-49bc-950c-f81f7213215b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:14.651358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe0e5b-05fc-49bc-950c-f81f7213215b None admin] PATH_INFO: `/v3/endpoints` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:14.652882 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe0e5b-05fc-49bc-950c-f81f7213215b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:14.656116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe0e5b-05fc-49bc-950c-f81f7213215b None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:14.714033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0bb3bd-d147-4ae3-bd63-2491cc2266e4 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:14.761880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a540d7b-b53e-4fb0-91d0-1a8e89a2e4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:14.762495 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a540d7b-b53e-4fb0-91d0-1a8e89a2e4b4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:14.762753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a540d7b-b53e-4fb0-91d0-1a8e89a2e4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:14.763068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a540d7b-b53e-4fb0-91d0-1a8e89a2e4b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:14.765703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:14.773008 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a540d7b-b53e-4fb0-91d0-1a8e89a2e4b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:14.774566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a540d7b-b53e-4fb0-91d0-1a8e89a2e4b4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:14.794051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 485/957] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 14:42:14 2022] POST /identity/v3/endpoints => generated 349 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:42:14.794051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe0e5b-05fc-49bc-950c-f81f7213215b None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-9a540d7b-b53e-4fb0-91d0-1a8e89a2e4b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:42:14.797126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:14.798937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:14.807346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:42:14.807346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:42:14.807346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:42:14.807346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:42:14.807346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:42:14.807346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:42:14.807346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:14.807346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:14.807346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 473/958] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:42:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:14.807346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc289ee5-ba76-4957-83a8-88b205de2d32 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:14.807346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc289ee5-ba76-4957-83a8-88b205de2d32 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:14.808135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc289ee5-ba76-4957-83a8-88b205de2d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:14.825412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-bc289ee5-ba76-4957-83a8-88b205de2d32 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:14.827277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-bc289ee5-ba76-4957-83a8-88b205de2d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:14.899542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:14.946327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 474/959] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:14.947958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc289ee5-ba76-4957-83a8-88b205de2d32 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:14.995890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe0e5b-05fc-49bc-950c-f81f7213215b None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:15.056483 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d26bd-2170-4829-b1b1-f8a7e7e132a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:15.057130 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d26bd-2170-4829-b1b1-f8a7e7e132a9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:15.057965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d26bd-2170-4829-b1b1-f8a7e7e132a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:15.057965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d26bd-2170-4829-b1b1-f8a7e7e132a9 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:15.058456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d26bd-2170-4829-b1b1-f8a7e7e132a9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:15.059180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d26bd-2170-4829-b1b1-f8a7e7e132a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:15.061747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d26bd-2170-4829-b1b1-f8a7e7e132a9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:15.106849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 486/960] 10.222.0.94 () {56 vars in 1101 bytes} [Tue Jul 26 14:42:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:15.107817 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d26bd-2170-4829-b1b1-f8a7e7e132a9 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:15.148908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc289ee5-ba76-4957-83a8-88b205de2d32 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:15.181752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:15.213550 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea25c3d-ddfb-449d-b390-5c0fdf866660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:15.214758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea25c3d-ddfb-449d-b390-5c0fdf866660 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:15.216088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea25c3d-ddfb-449d-b390-5c0fdf866660 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:15.217621 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea25c3d-ddfb-449d-b390-5c0fdf866660 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:15.220314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea25c3d-ddfb-449d-b390-5c0fdf866660 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:15.220636 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea25c3d-ddfb-449d-b390-5c0fdf866660 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:15.222378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea25c3d-ddfb-449d-b390-5c0fdf866660 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:15.249067 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 475/961] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:42:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:15.251962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea25c3d-ddfb-449d-b390-5c0fdf866660 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:15.290703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d26bd-2170-4829-b1b1-f8a7e7e132a9 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:15.305871 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:15.363358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6792e49-6123-4c41-a948-23491e81ff16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:15.364986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6792e49-6123-4c41-a948-23491e81ff16 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:15.364986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6792e49-6123-4c41-a948-23491e81ff16 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:15.364986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6792e49-6123-4c41-a948-23491e81ff16 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:15.365820 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6792e49-6123-4c41-a948-23491e81ff16 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:15.367368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6792e49-6123-4c41-a948-23491e81ff16 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-b6792e49-6123-4c41-a948-23491e81ff16 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:15.389330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:15.390584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:15.391778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 487/962] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:42:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:15.400448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6792e49-6123-4c41-a948-23491e81ff16 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:15.472083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea25c3d-ddfb-449d-b390-5c0fdf866660 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:15.532199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fd688-2570-4178-9408-30a21a0a3886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:15.533070 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fd688-2570-4178-9408-30a21a0a3886 None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:15.533454 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fd688-2570-4178-9408-30a21a0a3886 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:15.533865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fd688-2570-4178-9408-30a21a0a3886 None admin] PATH_INFO: `/v3/projects/a9b328041da24cedbc801b271d6e0419/users/5b24d190536a4953930432287dfbfb9b/roles/57e2f79cce1a4685848082d34f1eb041` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:15.537055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fd688-2570-4178-9408-30a21a0a3886 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9b328041da24cedbc801b271d6e0419, user_id=5b24d190536a4953930432287dfbfb9b, role_id=57e2f79cce1a4685848082d34f1eb041)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:15.538726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fd688-2570-4178-9408-30a21a0a3886 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:15.574633 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6792e49-6123-4c41-a948-23491e81ff16 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:15.619979 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5fdd7-a4af-4350-9b41-0ddf3bc82c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:15.620672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5fdd7-a4af-4350-9b41-0ddf3bc82c8b None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:15.621196 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5fdd7-a4af-4350-9b41-0ddf3bc82c8b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:15.621589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5fdd7-a4af-4350-9b41-0ddf3bc82c8b None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:15.622509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5fdd7-a4af-4350-9b41-0ddf3bc82c8b None admin] RBAC: Adding query filter params (name=compute) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:15.622930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5fdd7-a4af-4350-9b41-0ddf3bc82c8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:15.625482 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5fdd7-a4af-4350-9b41-0ddf3bc82c8b None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:15.631834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 488/963] 10.222.0.94 () {56 vars in 1112 bytes} [Tue Jul 26 14:42:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:15.637928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5fdd7-a4af-4350-9b41-0ddf3bc82c8b None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:15.685929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:15.713997 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 476/964] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:42:15 2022] PUT /identity/v3/projects/a9b328041da24cedbc801b271d6e0419/users/5b24d190536a4953930432287dfbfb9b/roles/57e2f79cce1a4685848082d34f1eb041 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:42:15.808115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5fdd7-a4af-4350-9b41-0ddf3bc82c8b None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:15.866910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974f106-6a50-4793-a176-fbb2476ff9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:15.868153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974f106-6a50-4793-a176-fbb2476ff9d0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:15.868865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974f106-6a50-4793-a176-fbb2476ff9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:15.869593 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974f106-6a50-4793-a176-fbb2476ff9d0 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:15.871177 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974f106-6a50-4793-a176-fbb2476ff9d0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:15.872114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974f106-6a50-4793-a176-fbb2476ff9d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:15.874371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974f106-6a50-4793-a176-fbb2476ff9d0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:15.884588 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 489/965] 10.222.0.94 () {56 vars in 1112 bytes} [Tue Jul 26 14:42:15 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:17.574981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f62b9-3af0-4d38-8684-c6e3522d6efa None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:17.575676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f62b9-3af0-4d38-8684-c6e3522d6efa None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:17.576022 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f62b9-3af0-4d38-8684-c6e3522d6efa None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:17.579072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 477/966] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:17.588114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe670f2-e7c6-4891-a4b3-281ece6e5c02 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:17.588924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe670f2-e7c6-4891-a4b3-281ece6e5c02 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:17.589097 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe670f2-e7c6-4891-a4b3-281ece6e5c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:17.623205 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-5fe670f2-e7c6-4891-a4b3-281ece6e5c02 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:17.626304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5fe670f2-e7c6-4891-a4b3-281ece6e5c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:17.767790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 490/967] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:17.803667 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f62b9-3af0-4d38-8684-c6e3522d6efa None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:17.887879 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:17.975449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f62b9-3af0-4d38-8684-c6e3522d6efa None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:18.035030 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75f674-7dc5-4edd-8eae-260e90bddd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:18.035810 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75f674-7dc5-4edd-8eae-260e90bddd2a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:18.036086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75f674-7dc5-4edd-8eae-260e90bddd2a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:18.036546 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75f674-7dc5-4edd-8eae-260e90bddd2a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:18.037301 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75f674-7dc5-4edd-8eae-260e90bddd2a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:18.038852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75f674-7dc5-4edd-8eae-260e90bddd2a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-4b75f674-7dc5-4edd-8eae-260e90bddd2a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:18.050789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:18.058136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:18.060117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 478/968] 10.222.0.94 () {56 vars in 1126 bytes} [Tue Jul 26 14:42:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:18.068829 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe670f2-e7c6-4891-a4b3-281ece6e5c02 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:18.132378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:18.283644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe670f2-e7c6-4891-a4b3-281ece6e5c02 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:18.338922 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea2348-b3c0-4bf5-b7bc-38f2d4b2fe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:18.339688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea2348-b3c0-4bf5-b7bc-38f2d4b2fe8e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:18.340009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea2348-b3c0-4bf5-b7bc-38f2d4b2fe8e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:18.340270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea2348-b3c0-4bf5-b7bc-38f2d4b2fe8e None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:18.341305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea2348-b3c0-4bf5-b7bc-38f2d4b2fe8e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:18.341716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea2348-b3c0-4bf5-b7bc-38f2d4b2fe8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:18.344116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea2348-b3c0-4bf5-b7bc-38f2d4b2fe8e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:18.350190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 491/969] 10.222.0.94 () {56 vars in 1122 bytes} [Tue Jul 26 14:42:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:18.356361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75f674-7dc5-4edd-8eae-260e90bddd2a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:18.497412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75f674-7dc5-4edd-8eae-260e90bddd2a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:18.501024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:18.545717 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a41b77-dbae-4a89-bb29-023c4f119ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:18.547637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a41b77-dbae-4a89-bb29-023c4f119ed7 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:18.548137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a41b77-dbae-4a89-bb29-023c4f119ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:18.548581 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a41b77-dbae-4a89-bb29-023c4f119ed7 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:18.549522 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a41b77-dbae-4a89-bb29-023c4f119ed7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:18.550027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a41b77-dbae-4a89-bb29-023c4f119ed7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:18.550415 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af5cc6-6b6f-4d15-a4a1-701b6d39babb None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:18.551259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af5cc6-6b6f-4d15-a4a1-701b6d39babb None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:18.551445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af5cc6-6b6f-4d15-a4a1-701b6d39babb None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:18.555937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a41b77-dbae-4a89-bb29-023c4f119ed7 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:18.556108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 492/970] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:42:18.563557 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 479/971] 10.222.0.94 () {56 vars in 1122 bytes} [Tue Jul 26 14:42:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:18.563809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a55cda-86a6-4c67-8e29-4db8addb4cc8 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:18.564308 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a55cda-86a6-4c67-8e29-4db8addb4cc8 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:18.564847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a55cda-86a6-4c67-8e29-4db8addb4cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:18.568872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a41b77-dbae-4a89-bb29-023c4f119ed7 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:18.598434 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-15a55cda-86a6-4c67-8e29-4db8addb4cc8 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:18.600705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-15a55cda-86a6-4c67-8e29-4db8addb4cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:18.723346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 493/972] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:42:18.728965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a41b77-dbae-4a89-bb29-023c4f119ed7 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:18.747883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a55cda-86a6-4c67-8e29-4db8addb4cc8 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:18.784284 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:18.788590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfe05b-4775-41fa-a5ea-5319c2f78662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:18.789230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfe05b-4775-41fa-a5ea-5319c2f78662 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:18.789518 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfe05b-4775-41fa-a5ea-5319c2f78662 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:18.789814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfe05b-4775-41fa-a5ea-5319c2f78662 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:18.790630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfe05b-4775-41fa-a5ea-5319c2f78662 None admin] RBAC: Adding query filter params (interface=admin, service_id=0ee57b62de0d492e85bb6677f367f84c, region_id=RegionOne) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:18.790950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfe05b-4775-41fa-a5ea-5319c2f78662 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:18.792484 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfe05b-4775-41fa-a5ea-5319c2f78662 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:18.799900 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 480/973] 10.222.0.94 () {56 vars in 1249 bytes} [Tue Jul 26 14:42:18 2022] GET /identity/v3/endpoints?service_id=0ee57b62de0d492e85bb6677f367f84c&interface=admin®ion_id=RegionOne => generated 193 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:18.809049 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfe05b-4775-41fa-a5ea-5319c2f78662 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:18.873700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:18.918515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a55cda-86a6-4c67-8e29-4db8addb4cc8 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:18.987004 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddb92d-bf9a-4c56-90ea-e313e0638b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:18.987881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddb92d-bf9a-4c56-90ea-e313e0638b58 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:18.988076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddb92d-bf9a-4c56-90ea-e313e0638b58 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:18.988483 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddb92d-bf9a-4c56-90ea-e313e0638b58 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:19.004857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddb92d-bf9a-4c56-90ea-e313e0638b58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-2bddb92d-bf9a-4c56-90ea-e313e0638b58 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:42:19.012448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:19.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:19.016766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:19.016766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:19.016766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:19.016766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:19.016766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:42:19.016766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:42:19.016766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:42:19.016766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:42:19.016766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:42:19.016766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:42:19.016766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:42:19.016766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:19.016766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 494/974] 10.222.0.94 () {56 vars in 1120 bytes} [Tue Jul 26 14:42:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:19.025360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddb92d-bf9a-4c56-90ea-e313e0638b58 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:19.026304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfe05b-4775-41fa-a5ea-5319c2f78662 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:19.071922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91c13d-6b66-4a40-9bd1-42a8425f6cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:19.072486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91c13d-6b66-4a40-9bd1-42a8425f6cd3 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:19.072818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91c13d-6b66-4a40-9bd1-42a8425f6cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:19.073126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91c13d-6b66-4a40-9bd1-42a8425f6cd3 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:19.073924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91c13d-6b66-4a40-9bd1-42a8425f6cd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:19.075428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91c13d-6b66-4a40-9bd1-42a8425f6cd3 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:19.082741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 481/975] 10.222.0.94 () {56 vars in 1087 bytes} [Tue Jul 26 14:42:18 2022] GET /identity/v3/services => generated 1497 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:42:19.084069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0d966-54bc-42bd-abf2-a83af13966c6 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:19.084384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0d966-54bc-42bd-abf2-a83af13966c6 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:19.084797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0d966-54bc-42bd-abf2-a83af13966c6 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:19.087501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 482/976] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:19.096872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647c2f-87b1-4a67-8194-caed0f2cdf05 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:19.097397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647c2f-87b1-4a67-8194-caed0f2cdf05 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:19.098014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647c2f-87b1-4a67-8194-caed0f2cdf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:19.130799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-7f647c2f-87b1-4a67-8194-caed0f2cdf05 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:19.132804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-7f647c2f-87b1-4a67-8194-caed0f2cdf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:19.182140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:19.207871 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddb92d-bf9a-4c56-90ea-e313e0638b58 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:19.275931 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 483/977] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:19.280771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44fd17-a306-440f-8fdd-37757e5b5343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:19.281680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44fd17-a306-440f-8fdd-37757e5b5343 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:19.282366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44fd17-a306-440f-8fdd-37757e5b5343 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:19.282814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44fd17-a306-440f-8fdd-37757e5b5343 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:19.284593 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44fd17-a306-440f-8fdd-37757e5b5343 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:19.285095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44fd17-a306-440f-8fdd-37757e5b5343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:19.287617 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44fd17-a306-440f-8fdd-37757e5b5343 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:19.303669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f647c2f-87b1-4a67-8194-caed0f2cdf05 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:19.305378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 495/978] 10.222.0.94 () {56 vars in 1115 bytes} [Tue Jul 26 14:42:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:19.313476 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44fd17-a306-440f-8fdd-37757e5b5343 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:19.346362 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:19.476255 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f647c2f-87b1-4a67-8194-caed0f2cdf05 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:19.478772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44fd17-a306-440f-8fdd-37757e5b5343 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:19.529344 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86214fd7-0310-4377-a34e-19a700af4ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:19.529990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86214fd7-0310-4377-a34e-19a700af4ab4 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:19.530283 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86214fd7-0310-4377-a34e-19a700af4ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:19.530690 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86214fd7-0310-4377-a34e-19a700af4ab4 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:19.531580 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86214fd7-0310-4377-a34e-19a700af4ab4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:19.533175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86214fd7-0310-4377-a34e-19a700af4ab4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:19.536276 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97c18e-48f4-4df8-bd04-23ae9cea290a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:19.537345 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97c18e-48f4-4df8-bd04-23ae9cea290a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:19.537840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97c18e-48f4-4df8-bd04-23ae9cea290a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:19.538404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97c18e-48f4-4df8-bd04-23ae9cea290a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:19.553301 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97c18e-48f4-4df8-bd04-23ae9cea290a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:19.554783 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97c18e-48f4-4df8-bd04-23ae9cea290a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:19.564312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-3a97c18e-48f4-4df8-bd04-23ae9cea290a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:42:19.573501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:19.575006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:19.576051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:42:19.576051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:42:19.576051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:19.576051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:19.576051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 496/979] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:19.582498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97c18e-48f4-4df8-bd04-23ae9cea290a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:19.649051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 484/980] 10.222.0.94 () {60 vars in 1141 bytes} [Tue Jul 26 14:42:19 2022] POST /identity/v3/services => generated 242 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:42:19.752327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97c18e-48f4-4df8-bd04-23ae9cea290a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:19.844670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649deb8-8aa2-448a-ade7-5bd5783a97d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:19.845660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649deb8-8aa2-448a-ade7-5bd5783a97d7 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:19.846158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649deb8-8aa2-448a-ade7-5bd5783a97d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:19.846653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649deb8-8aa2-448a-ade7-5bd5783a97d7 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:19.848032 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0649deb8-8aa2-448a-ade7-5bd5783a97d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:19.848607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0649deb8-8aa2-448a-ade7-5bd5783a97d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:19.851290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0649deb8-8aa2-448a-ade7-5bd5783a97d7 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:19.869890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 497/981] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:19.876097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86214fd7-0310-4377-a34e-19a700af4ab4 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:19.996713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:20.024048 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86214fd7-0310-4377-a34e-19a700af4ab4 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:20.097813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede4237-44c1-4eab-be62-0559aca6edb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:20.098948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede4237-44c1-4eab-be62-0559aca6edb9 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:20.099837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede4237-44c1-4eab-be62-0559aca6edb9 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:20.100412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede4237-44c1-4eab-be62-0559aca6edb9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:20.117406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede4237-44c1-4eab-be62-0559aca6edb9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:20.121086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede4237-44c1-4eab-be62-0559aca6edb9 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-eede4237-44c1-4eab-be62-0559aca6edb9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:42:20.150370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:20.151579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:20.152707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:20.152707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:20.152707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:42:20.152707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:42:20.152707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:42:20.152707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:42:20.152707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:42:20.152707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:42:20.152707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 14:42:20.152707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:20.153926 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 485/982] 10.222.0.94 () {56 vars in 1117 bytes} [Tue Jul 26 14:42:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:20.163371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649deb8-8aa2-448a-ade7-5bd5783a97d7 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:20.286051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:20.348224 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649deb8-8aa2-448a-ade7-5bd5783a97d7 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:20.401447 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92cc39-962d-4b14-b729-e216ac4bdf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:20.402176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92cc39-962d-4b14-b729-e216ac4bdf4e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:20.402482 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92cc39-962d-4b14-b729-e216ac4bdf4e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:20.402813 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92cc39-962d-4b14-b729-e216ac4bdf4e None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:20.403704 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92cc39-962d-4b14-b729-e216ac4bdf4e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:20.404020 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92cc39-962d-4b14-b729-e216ac4bdf4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:20.406538 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92cc39-962d-4b14-b729-e216ac4bdf4e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:20.432361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 498/983] 10.222.0.94 () {56 vars in 1149 bytes} [Tue Jul 26 14:42:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:20.439514 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede4237-44c1-4eab-be62-0559aca6edb9 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:20.578560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede4237-44c1-4eab-be62-0559aca6edb9 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:20.626760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f455e0-0436-4328-bac2-2da4f4f89c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:20.627548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f455e0-0436-4328-bac2-2da4f4f89c54 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:20.627876 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f455e0-0436-4328-bac2-2da4f4f89c54 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:20.628186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f455e0-0436-4328-bac2-2da4f4f89c54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:20.652380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:20.730914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f455e0-0436-4328-bac2-2da4f4f89c54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:20.732636 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f455e0-0436-4328-bac2-2da4f4f89c54 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:20.882585 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 486/984] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:42:20 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:42:20.888163 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92cc39-962d-4b14-b729-e216ac4bdf4e None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:21.036425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92cc39-962d-4b14-b729-e216ac4bdf4e None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:21.056013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:21.084153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb86cb-e84c-4fed-b418-c8d99aaa0556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:21.085040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb86cb-e84c-4fed-b418-c8d99aaa0556 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:21.085323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb86cb-e84c-4fed-b418-c8d99aaa0556 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:21.085601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb86cb-e84c-4fed-b418-c8d99aaa0556 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:21.101291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb86cb-e84c-4fed-b418-c8d99aaa0556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:21.102955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb86cb-e84c-4fed-b418-c8d99aaa0556 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-a1cb86cb-e84c-4fed-b418-c8d99aaa0556 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:42:21.124789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:21.127696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:21.129504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:42:21.129504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:42:21.129504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:21.129504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:21.134904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 499/985] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:21.135148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f455e0-0436-4328-bac2-2da4f4f89c54 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:21.230299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:21.357193 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f455e0-0436-4328-bac2-2da4f4f89c54 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:21.428418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6f0f1-4fd2-411f-8d12-153da820c331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:21.429307 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6f0f1-4fd2-411f-8d12-153da820c331 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:21.429747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6f0f1-4fd2-411f-8d12-153da820c331 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:21.430195 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6f0f1-4fd2-411f-8d12-153da820c331 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:21.431366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6f0f1-4fd2-411f-8d12-153da820c331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:21.431931 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6f0f1-4fd2-411f-8d12-153da820c331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:21.434843 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6f0f1-4fd2-411f-8d12-153da820c331 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:21.460698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 487/986] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:21.470579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb86cb-e84c-4fed-b418-c8d99aaa0556 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:21.679360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb86cb-e84c-4fed-b418-c8d99aaa0556 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:21.749635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25852297-3e4b-437a-84bd-3fe08c3f7b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:21.750766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25852297-3e4b-437a-84bd-3fe08c3f7b5f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:21.751236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25852297-3e4b-437a-84bd-3fe08c3f7b5f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:21.751769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25852297-3e4b-437a-84bd-3fe08c3f7b5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:21.765908 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25852297-3e4b-437a-84bd-3fe08c3f7b5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:21.768816 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25852297-3e4b-437a-84bd-3fe08c3f7b5f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-25852297-3e4b-437a-84bd-3fe08c3f7b5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:42:21.793179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:21.794952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:21.797189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:42:21.797189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:42:21.797189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:42:21.797189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:42:21.797189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:42:21.797189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:42:21.797189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:21.797189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:21.799882 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 500/987] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:42:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:21.805587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6f0f1-4fd2-411f-8d12-153da820c331 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:22.025492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6f0f1-4fd2-411f-8d12-153da820c331 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:22.042547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:22.083310 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4ef6f-c4bc-4d22-b475-a34ffd065ac8 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:22.084699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4ef6f-c4bc-4d22-b475-a34ffd065ac8 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:22.085614 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4ef6f-c4bc-4d22-b475-a34ffd065ac8 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:22.088890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 501/988] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:22.096635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e02f9-3c33-49a6-9278-2f840f3e1b9f None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:22.097138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e02f9-3c33-49a6-9278-2f840f3e1b9f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:22.097571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e02f9-3c33-49a6-9278-2f840f3e1b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:22.099687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a0230-db57-4b5c-a46d-6c966226daa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:22.100760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a0230-db57-4b5c-a46d-6c966226daa0 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:22.101241 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a0230-db57-4b5c-a46d-6c966226daa0 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:22.102299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a0230-db57-4b5c-a46d-6c966226daa0 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:22.103722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a0230-db57-4b5c-a46d-6c966226daa0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:22.104819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a0230-db57-4b5c-a46d-6c966226daa0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:22.109093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a0230-db57-4b5c-a46d-6c966226daa0 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:22.122265 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-776e02f9-3c33-49a6-9278-2f840f3e1b9f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:22.123681 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-776e02f9-3c33-49a6-9278-2f840f3e1b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:22.147554 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 488/989] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:42:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:22.151019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a0230-db57-4b5c-a46d-6c966226daa0 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:22.160142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:22.237161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 502/990] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:42:22.239349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864acdc-d186-4cf7-9c9a-da034058ebdb None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:22.239855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864acdc-d186-4cf7-9c9a-da034058ebdb None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:22.240290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864acdc-d186-4cf7-9c9a-da034058ebdb None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:22.243746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 503/991] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:22.252924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe74a36-e8d7-4107-a33e-da3a280bd273 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:22.253390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe74a36-e8d7-4107-a33e-da3a280bd273 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:22.253857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe74a36-e8d7-4107-a33e-da3a280bd273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:22.279359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-dbe74a36-e8d7-4107-a33e-da3a280bd273 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:22.280875 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-dbe74a36-e8d7-4107-a33e-da3a280bd273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:22.318258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a0230-db57-4b5c-a46d-6c966226daa0 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:22.380924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578fb07-a22d-4f6d-9297-cf63e51d8ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:22.381702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578fb07-a22d-4f6d-9297-cf63e51d8ea5 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:22.382104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578fb07-a22d-4f6d-9297-cf63e51d8ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:22.382475 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578fb07-a22d-4f6d-9297-cf63e51d8ea5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:22.383852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578fb07-a22d-4f6d-9297-cf63e51d8ea5 None admin] RBAC: Adding query filter params (role.id=57e2f79cce1a4685848082d34f1eb041, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=5b24d190536a4953930432287dfbfb9b) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:22.384297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578fb07-a22d-4f6d-9297-cf63e51d8ea5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:22.386036 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578fb07-a22d-4f6d-9297-cf63e51d8ea5 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:22.393641 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 504/992] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:22.402537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe74a36-e8d7-4107-a33e-da3a280bd273 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:22.402827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 489/993] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:42:22 2022] GET /identity/v3/role_assignments?user.id=5b24d190536a4953930432287dfbfb9b&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=57e2f79cce1a4685848082d34f1eb041 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:22.434713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578fb07-a22d-4f6d-9297-cf63e51d8ea5 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:22.496262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:22.573430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:22.605462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe74a36-e8d7-4107-a33e-da3a280bd273 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:22.626817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578fb07-a22d-4f6d-9297-cf63e51d8ea5 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:22.667339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a249e11-e2ff-4470-87c0-cca09c0ed721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:22.669177 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a249e11-e2ff-4470-87c0-cca09c0ed721 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:22.669913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a249e11-e2ff-4470-87c0-cca09c0ed721 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:22.670390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a249e11-e2ff-4470-87c0-cca09c0ed721 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:22.673396 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a249e11-e2ff-4470-87c0-cca09c0ed721 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:22.674729 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a249e11-e2ff-4470-87c0-cca09c0ed721 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-8a249e11-e2ff-4470-87c0-cca09c0ed721 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:22.694216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:22.696286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:22.698909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:22.698909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:22.698909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:22.698909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:22.698909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:22.698909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:22.698909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:22.698909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:22.698909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:22.698909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:22.698909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:22.698909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:22.698909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 14:42:22.698909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:22.700681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0588a-f7cd-462d-9cad-759d160a6754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:22.700681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0588a-f7cd-462d-9cad-759d160a6754 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:22.700681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0588a-f7cd-462d-9cad-759d160a6754 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:22.700681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0588a-f7cd-462d-9cad-759d160a6754 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:22.701246 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 505/994] 10.222.0.94 () {56 vars in 1126 bytes} [Tue Jul 26 14:42:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:22.701405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0588a-f7cd-462d-9cad-759d160a6754 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:22.701405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb0588a-f7cd-462d-9cad-759d160a6754 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:22.703798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a249e11-e2ff-4470-87c0-cca09c0ed721 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-7cb0588a-f7cd-462d-9cad-759d160a6754 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:22.718635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:22.722687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:22.726176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 490/995] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:42:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:22.731135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0588a-f7cd-462d-9cad-759d160a6754 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:22.890272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:22.903576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0588a-f7cd-462d-9cad-759d160a6754 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:22.910427 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a249e11-e2ff-4470-87c0-cca09c0ed721 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:22.967330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf0c47-4930-437c-8cbd-57e2c33c6d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:22.968570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf0c47-4930-437c-8cbd-57e2c33c6d02 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:22.969037 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf0c47-4930-437c-8cbd-57e2c33c6d02 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:22.969512 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf0c47-4930-437c-8cbd-57e2c33c6d02 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:22.969871 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600ea76-3c08-4a5d-a527-294ada6f0941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:22.970619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf0c47-4930-437c-8cbd-57e2c33c6d02 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:22.970867 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600ea76-3c08-4a5d-a527-294ada6f0941 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:22.971361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600ea76-3c08-4a5d-a527-294ada6f0941 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:22.971555 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf0c47-4930-437c-8cbd-57e2c33c6d02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:22.971901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600ea76-3c08-4a5d-a527-294ada6f0941 None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:22.973099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600ea76-3c08-4a5d-a527-294ada6f0941 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:22.973763 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600ea76-3c08-4a5d-a527-294ada6f0941 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:22.974119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf0c47-4930-437c-8cbd-57e2c33c6d02 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:22.976423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600ea76-3c08-4a5d-a527-294ada6f0941 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:22.982595 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 491/996] 10.222.0.94 () {56 vars in 1112 bytes} [Tue Jul 26 14:42:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:22.991544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 506/997] 10.222.0.94 () {56 vars in 1122 bytes} [Tue Jul 26 14:42:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:22.993208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf0c47-4930-437c-8cbd-57e2c33c6d02 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:22.993828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600ea76-3c08-4a5d-a527-294ada6f0941 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:23.026235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:23.168092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf0c47-4930-437c-8cbd-57e2c33c6d02 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:23.194537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600ea76-3c08-4a5d-a527-294ada6f0941 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:23.237296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956dd395-aeab-4bb7-aa41-821704fa63cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:23.238371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dd395-aeab-4bb7-aa41-821704fa63cd None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:23.238870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dd395-aeab-4bb7-aa41-821704fa63cd None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:23.239355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dd395-aeab-4bb7-aa41-821704fa63cd None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:23.240688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956dd395-aeab-4bb7-aa41-821704fa63cd None admin] RBAC: Adding query filter params (type=compute) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:23.241270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956dd395-aeab-4bb7-aa41-821704fa63cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:23.243974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956dd395-aeab-4bb7-aa41-821704fa63cd None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:23.260481 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 492/998] 10.222.0.94 () {56 vars in 1112 bytes} [Tue Jul 26 14:42:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:23.262520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956dd395-aeab-4bb7-aa41-821704fa63cd None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:23.269210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffa0a7-af3d-4e66-964c-f640403d2bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:23.269210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffa0a7-af3d-4e66-964c-f640403d2bde None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:23.269210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffa0a7-af3d-4e66-964c-f640403d2bde None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:23.269210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffa0a7-af3d-4e66-964c-f640403d2bde None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:23.269782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ffa0a7-af3d-4e66-964c-f640403d2bde None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:23.269782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ffa0a7-af3d-4e66-964c-f640403d2bde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:23.272583 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ffa0a7-af3d-4e66-964c-f640403d2bde None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:23.295077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 507/999] 10.222.0.94 () {56 vars in 1122 bytes} [Tue Jul 26 14:42:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:23.297163 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffa0a7-af3d-4e66-964c-f640403d2bde None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:23.307320 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:23.436444 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:23.467521 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956dd395-aeab-4bb7-aa41-821704fa63cd None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:23.486849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffa0a7-af3d-4e66-964c-f640403d2bde None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:23.533284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777e234-11f0-4198-b770-9532ebfadbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:23.535143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777e234-11f0-4198-b770-9532ebfadbe3 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:23.535786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777e234-11f0-4198-b770-9532ebfadbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:23.536265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777e234-11f0-4198-b770-9532ebfadbe3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:23.537875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777e234-11f0-4198-b770-9532ebfadbe3 None admin] RBAC: Adding query filter params (interface=public, service_id=79e5bc7387374309a1747a1b3b888926, region_id=RegionOne) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:23.538581 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777e234-11f0-4198-b770-9532ebfadbe3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:23.541358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777e234-11f0-4198-b770-9532ebfadbe3 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:23.557127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 493/1000] 10.222.0.94 () {56 vars in 1251 bytes} [Tue Jul 26 14:42:23 2022] GET /identity/v3/endpoints?service_id=79e5bc7387374309a1747a1b3b888926&interface=public®ion_id=RegionOne => generated 194 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:23.559142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777e234-11f0-4198-b770-9532ebfadbe3 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:23.568753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f89a28-76b4-473b-be0c-510e1c5acb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:23.569661 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f89a28-76b4-473b-be0c-510e1c5acb08 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:23.570145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f89a28-76b4-473b-be0c-510e1c5acb08 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:23.570605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f89a28-76b4-473b-be0c-510e1c5acb08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:23.572004 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f89a28-76b4-473b-be0c-510e1c5acb08 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:23.574607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f89a28-76b4-473b-be0c-510e1c5acb08 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:23.688960 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 508/1001] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 14:42:23 2022] POST /identity/v3/endpoints => generated 325 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:42:23.716873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:23.739576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777e234-11f0-4198-b770-9532ebfadbe3 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:23.819940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9209a-3447-4f17-81eb-18649e7c2982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:23.821069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9209a-3447-4f17-81eb-18649e7c2982 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:23.821583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9209a-3447-4f17-81eb-18649e7c2982 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:23.822098 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9209a-3447-4f17-81eb-18649e7c2982 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:23.823405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9209a-3447-4f17-81eb-18649e7c2982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:23.826201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9209a-3447-4f17-81eb-18649e7c2982 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:23.840612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 494/1002] 10.222.0.94 () {56 vars in 1087 bytes} [Tue Jul 26 14:42:23 2022] GET /identity/v3/services => generated 1727 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 14:42:24.984474 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcdda3-6b35-49b6-ab7f-e020cf38ac7e None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:24.985286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcdda3-6b35-49b6-ab7f-e020cf38ac7e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:24.985445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcdda3-6b35-49b6-ab7f-e020cf38ac7e None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:24.988184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 509/1003] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:24.996118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71001039-986f-4f5b-a608-f5be216262b0 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:24.996895 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71001039-986f-4f5b-a608-f5be216262b0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:24.996895 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71001039-986f-4f5b-a608-f5be216262b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:25.026391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-71001039-986f-4f5b-a608-f5be216262b0 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:25.028470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-71001039-986f-4f5b-a608-f5be216262b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:25.169093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 495/1004] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:24 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:42:25.203014 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcdda3-6b35-49b6-ab7f-e020cf38ac7e None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:25.373869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcdda3-6b35-49b6-ab7f-e020cf38ac7e None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:25.419940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8d55a-22d1-4e9f-9ec3-9c05e73ddd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:25.421379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8d55a-22d1-4e9f-9ec3-9c05e73ddd81 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:25.421801 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8d55a-22d1-4e9f-9ec3-9c05e73ddd81 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:25.422318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8d55a-22d1-4e9f-9ec3-9c05e73ddd81 None admin] PATH_INFO: `/v3/services/image` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:25.423160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac8d55a-22d1-4e9f-9ec3-9c05e73ddd81 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:25.424920 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac8d55a-22d1-4e9f-9ec3-9c05e73ddd81 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:25.429059 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-4ac8d55a-22d1-4e9f-9ec3-9c05e73ddd81 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:25.439366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:25.441712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:25.444461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:25.444461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:25.444461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:25.444461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:25.444461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:25.444461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:25.444461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:25.444461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:25.444461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:25.444461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:25.444461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:25.444461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:25.444461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:42:25.444461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:25.452625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 510/1005] 10.222.0.94 () {56 vars in 1105 bytes} [Tue Jul 26 14:42:25 2022] GET /identity/v3/services/image => generated 86 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:25.452890 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71001039-986f-4f5b-a608-f5be216262b0 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:25.611396 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:25.657176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71001039-986f-4f5b-a608-f5be216262b0 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:25.723414 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f3d8f-7cbe-4ef8-90aa-710ac53477ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:25.724707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f3d8f-7cbe-4ef8-90aa-710ac53477ea None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:25.725005 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f3d8f-7cbe-4ef8-90aa-710ac53477ea None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:25.725489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f3d8f-7cbe-4ef8-90aa-710ac53477ea None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:25.726811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f3d8f-7cbe-4ef8-90aa-710ac53477ea None admin] RBAC: Adding query filter params (name=image) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:25.727364 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f3d8f-7cbe-4ef8-90aa-710ac53477ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:25.730187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f3d8f-7cbe-4ef8-90aa-710ac53477ea None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:25.746208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 496/1006] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:25.748104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8d55a-22d1-4e9f-9ec3-9c05e73ddd81 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:25.951456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8d55a-22d1-4e9f-9ec3-9c05e73ddd81 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:26.013951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36513684-cae4-40c8-a243-01a0a21c681f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:26.015444 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36513684-cae4-40c8-a243-01a0a21c681f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:26.015875 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36513684-cae4-40c8-a243-01a0a21c681f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:26.016245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36513684-cae4-40c8-a243-01a0a21c681f None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:26.017285 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36513684-cae4-40c8-a243-01a0a21c681f None admin] RBAC: Adding query filter params (type=image) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:26.017735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36513684-cae4-40c8-a243-01a0a21c681f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:26.020112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36513684-cae4-40c8-a243-01a0a21c681f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:26.035565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 511/1007] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:26.152936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db416a-aefe-4b8e-892f-7166f42c69ca None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:26.153479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db416a-aefe-4b8e-892f-7166f42c69ca None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:26.153479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db416a-aefe-4b8e-892f-7166f42c69ca None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:26.155397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 497/1008] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:26.162915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7310c-ef20-4272-b392-533b8d701d28 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:26.163422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7310c-ef20-4272-b392-533b8d701d28 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:26.163996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7310c-ef20-4272-b392-533b8d701d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:26.195999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-a5f7310c-ef20-4272-b392-533b8d701d28 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:26.198097 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a5f7310c-ef20-4272-b392-533b8d701d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:26.298897 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d02e9-addf-4911-ad9c-94e5498a4126 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:26.299313 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d02e9-addf-4911-ad9c-94e5498a4126 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:26.299448 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d02e9-addf-4911-ad9c-94e5498a4126 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:26.300643 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:26.301003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 498/1009] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:26.307916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b852560-237b-4ea4-afb3-662d899eb2ef None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:26.308180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b852560-237b-4ea4-afb3-662d899eb2ef None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:26.308528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b852560-237b-4ea4-afb3-662d899eb2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:26.315119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 512/1010] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:26 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:26.334430 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-8b852560-237b-4ea4-afb3-662d899eb2ef None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:26.336302 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8b852560-237b-4ea4-afb3-662d899eb2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:26.341895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7310c-ef20-4272-b392-533b8d701d28 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:26.439444 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 499/1011] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:26 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:42:26.464683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b852560-237b-4ea4-afb3-662d899eb2ef None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:26.497212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7310c-ef20-4272-b392-533b8d701d28 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:26.544990 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91420427-ce20-41cb-af41-afc6ccdab9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:26.546140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91420427-ce20-41cb-af41-afc6ccdab9ee None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:26.546418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91420427-ce20-41cb-af41-afc6ccdab9ee None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:26.546725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91420427-ce20-41cb-af41-afc6ccdab9ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:26.548132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91420427-ce20-41cb-af41-afc6ccdab9ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:26.549610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91420427-ce20-41cb-af41-afc6ccdab9ee None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:26.558916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 513/1012] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:26.560829 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91420427-ce20-41cb-af41-afc6ccdab9ee None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:26.563790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:26.604705 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b852560-237b-4ea4-afb3-662d899eb2ef None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:26.666041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:26.672887 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5843e3-8e14-4ddd-b05b-0798a37e531f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:26.673744 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5843e3-8e14-4ddd-b05b-0798a37e531f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:26.674206 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5843e3-8e14-4ddd-b05b-0798a37e531f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:26.674669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5843e3-8e14-4ddd-b05b-0798a37e531f None admin] PATH_INFO: `/v3/services/compute` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:26.675648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5843e3-8e14-4ddd-b05b-0798a37e531f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:26.677455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5843e3-8e14-4ddd-b05b-0798a37e531f None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-ed5843e3-8e14-4ddd-b05b-0798a37e531f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:26.689172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:26.691173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:26.693545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 500/1013] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:42:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:26.699791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5843e3-8e14-4ddd-b05b-0798a37e531f None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:26.748330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91420427-ce20-41cb-af41-afc6ccdab9ee None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:26.816608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad301b4-6919-4a50-9d47-1ff699c24c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:26.817149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad301b4-6919-4a50-9d47-1ff699c24c18 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:26.817430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad301b4-6919-4a50-9d47-1ff699c24c18 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:26.817707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad301b4-6919-4a50-9d47-1ff699c24c18 None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:26.818550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad301b4-6919-4a50-9d47-1ff699c24c18 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:26.820418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad301b4-6919-4a50-9d47-1ff699c24c18 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:26.863137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5843e3-8e14-4ddd-b05b-0798a37e531f None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:26.910027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9de651-de75-4386-a0b7-59d1f63d845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:26.910675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9de651-de75-4386-a0b7-59d1f63d845d None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:26.910957 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9de651-de75-4386-a0b7-59d1f63d845d None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:26.911236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9de651-de75-4386-a0b7-59d1f63d845d None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:26.911982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9de651-de75-4386-a0b7-59d1f63d845d None admin] RBAC: Adding query filter params (name=compute) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:26.912367 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9de651-de75-4386-a0b7-59d1f63d845d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:26.913858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9de651-de75-4386-a0b7-59d1f63d845d None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:26.915274 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:26.921663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 501/1014] 10.222.0.94 () {56 vars in 1112 bytes} [Tue Jul 26 14:42:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:26.928509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9de651-de75-4386-a0b7-59d1f63d845d None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:27.024013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 514/1015] 10.222.0.94 () {60 vars in 1141 bytes} [Tue Jul 26 14:42:26 2022] POST /identity/v3/projects => generated 311 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:42:27.091368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9de651-de75-4386-a0b7-59d1f63d845d None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:27.167167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a1260-db67-48e5-b2be-532d902d01e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:27.168175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a1260-db67-48e5-b2be-532d902d01e8 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:27.168636 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a1260-db67-48e5-b2be-532d902d01e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:27.169096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a1260-db67-48e5-b2be-532d902d01e8 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:27.170351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a1260-db67-48e5-b2be-532d902d01e8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:27.171011 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a1260-db67-48e5-b2be-532d902d01e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:27.173645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a1260-db67-48e5-b2be-532d902d01e8 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:27.188680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 502/1016] 10.222.0.94 () {56 vars in 1112 bytes} [Tue Jul 26 14:42:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:27.191749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad301b4-6919-4a50-9d47-1ff699c24c18 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:27.414900 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad301b4-6919-4a50-9d47-1ff699c24c18 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:27.444836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:27.473347 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cf601-7485-4125-b4a3-144026dc8e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:27.474037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cf601-7485-4125-b4a3-144026dc8e86 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:27.474311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cf601-7485-4125-b4a3-144026dc8e86 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:27.474581 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cf601-7485-4125-b4a3-144026dc8e86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:27.475366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7cf601-7485-4125-b4a3-144026dc8e86 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:27.476972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7cf601-7485-4125-b4a3-144026dc8e86 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:27.691606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 515/1017] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 14:42:27 2022] POST /identity/v3/endpoints => generated 334 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:42:28.568766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6aa7da-1a07-48cc-89e7-2d090c7c0a5a None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:28.569586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6aa7da-1a07-48cc-89e7-2d090c7c0a5a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:28.569739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6aa7da-1a07-48cc-89e7-2d090c7c0a5a None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:28.572580 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 503/1018] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:28.582313 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebca27-03e0-4d70-af00-058160a4ab20 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:28.583081 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebca27-03e0-4d70-af00-058160a4ab20 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:28.583335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebca27-03e0-4d70-af00-058160a4ab20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:28.614883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c2ebca27-03e0-4d70-af00-058160a4ab20 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:28.617023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c2ebca27-03e0-4d70-af00-058160a4ab20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:28.759507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 516/1019] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:28 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:42:28.785001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6aa7da-1a07-48cc-89e7-2d090c7c0a5a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:28.880619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:28.934581 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6aa7da-1a07-48cc-89e7-2d090c7c0a5a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:28.977407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23e467-968a-4336-9b1c-3114c832689b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:28.978127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23e467-968a-4336-9b1c-3114c832689b None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:28.978434 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23e467-968a-4336-9b1c-3114c832689b None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:28.978721 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23e467-968a-4336-9b1c-3114c832689b None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:28.979449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23e467-968a-4336-9b1c-3114c832689b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:28.980862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23e467-968a-4336-9b1c-3114c832689b None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:29.178575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 504/1020] 10.222.0.94 () {60 vars in 1141 bytes} [Tue Jul 26 14:42:28 2022] POST /identity/v3/services => generated 242 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:42:29.734708 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8129f1e-6e67-4e90-baa3-f58f3b03479e None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:29.735604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8129f1e-6e67-4e90-baa3-f58f3b03479e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:29.735604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8129f1e-6e67-4e90-baa3-f58f3b03479e None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:29.737385 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 517/1021] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:29.743973 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8fa99-bc09-4729-b6e1-94ce51f0c9f3 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:29.744811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8fa99-bc09-4729-b6e1-94ce51f0c9f3 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:29.744963 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8fa99-bc09-4729-b6e1-94ce51f0c9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:29.769683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-b3d8fa99-bc09-4729-b6e1-94ce51f0c9f3 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:29.770989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b3d8fa99-bc09-4729-b6e1-94ce51f0c9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:29.870534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 505/1022] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:42:29.896780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8129f1e-6e67-4e90-baa3-f58f3b03479e None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:30.027665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:30.038421 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8129f1e-6e67-4e90-baa3-f58f3b03479e None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:30.088810 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e89568-e504-4c5f-b758-8ed223d7cc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:30.089495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e89568-e504-4c5f-b758-8ed223d7cc4e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:30.090124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e89568-e504-4c5f-b758-8ed223d7cc4e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:30.090451 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e89568-e504-4c5f-b758-8ed223d7cc4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:30.091883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e89568-e504-4c5f-b758-8ed223d7cc4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:30.094157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e89568-e504-4c5f-b758-8ed223d7cc4e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:30.096290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 518/1023] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:30.102931 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8fa99-bc09-4729-b6e1-94ce51f0c9f3 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:30.201340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:30.320922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8fa99-bc09-4729-b6e1-94ce51f0c9f3 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:30.392068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f49bb-e091-4ebe-912d-6f3290407c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:30.393176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f49bb-e091-4ebe-912d-6f3290407c55 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:30.393673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f49bb-e091-4ebe-912d-6f3290407c55 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:30.394168 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f49bb-e091-4ebe-912d-6f3290407c55 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:30.395573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f49bb-e091-4ebe-912d-6f3290407c55 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:30.398514 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f49bb-e091-4ebe-912d-6f3290407c55 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:30.551822 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 506/1024] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:42:30 2022] POST /identity/v3/users => generated 285 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:42:30.690511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58262805-f64c-4f50-b470-0999444fec6d None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:30.690758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58262805-f64c-4f50-b470-0999444fec6d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:30.691051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58262805-f64c-4f50-b470-0999444fec6d None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:30.693402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 519/1025] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:42:30.699578 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122605d4-0dce-458d-9dc5-451a1ec3ef02 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:30.699854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122605d4-0dce-458d-9dc5-451a1ec3ef02 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:30.700129 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122605d4-0dce-458d-9dc5-451a1ec3ef02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:30.724768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-122605d4-0dce-458d-9dc5-451a1ec3ef02 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:30.726107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-122605d4-0dce-458d-9dc5-451a1ec3ef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:30.831950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 507/1026] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:30.870920 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58262805-f64c-4f50-b470-0999444fec6d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:31.035688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58262805-f64c-4f50-b470-0999444fec6d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:31.086500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a26cfa-837a-4fe9-9385-45c48dd3ad68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:31.087212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a26cfa-837a-4fe9-9385-45c48dd3ad68 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:31.087531 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a26cfa-837a-4fe9-9385-45c48dd3ad68 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:31.087796 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a26cfa-837a-4fe9-9385-45c48dd3ad68 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:31.101826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a26cfa-837a-4fe9-9385-45c48dd3ad68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-18a26cfa-837a-4fe9-9385-45c48dd3ad68 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:42:31.109082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:31.111769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:31.113450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:31.113450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:31.113450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:31.113450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:31.113450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:42:31.113450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:42:31.113450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:42:31.113450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:42:31.113450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:42:31.113450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:42:31.113450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:42:31.113450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:31.113450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 520/1027] 10.222.0.94 () {56 vars in 1120 bytes} [Tue Jul 26 14:42:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:31.119040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122605d4-0dce-458d-9dc5-451a1ec3ef02 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:31.183853 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:31.300000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122605d4-0dce-458d-9dc5-451a1ec3ef02 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:31.363843 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c34c1-cec6-4ccf-8717-882e85978ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:31.364965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c34c1-cec6-4ccf-8717-882e85978ae4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:31.365444 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c34c1-cec6-4ccf-8717-882e85978ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:31.365921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c34c1-cec6-4ccf-8717-882e85978ae4 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:31.367209 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c34c1-cec6-4ccf-8717-882e85978ae4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:31.367787 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c34c1-cec6-4ccf-8717-882e85978ae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:31.370575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c34c1-cec6-4ccf-8717-882e85978ae4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:31.388965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 508/1028] 10.222.0.94 () {56 vars in 1115 bytes} [Tue Jul 26 14:42:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:31.396031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a26cfa-837a-4fe9-9385-45c48dd3ad68 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:31.449478 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:31.607274 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a26cfa-837a-4fe9-9385-45c48dd3ad68 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:31.643220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e6dab-ad05-47d3-8a55-886f7a417e06 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:31.643641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e6dab-ad05-47d3-8a55-886f7a417e06 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:31.643911 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e6dab-ad05-47d3-8a55-886f7a417e06 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:31.645512 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 509/1029] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:42:31.652047 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60ced4-88e8-4ffb-9ad9-d4eef42a5cc8 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:31.652525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60ced4-88e8-4ffb-9ad9-d4eef42a5cc8 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:31.653016 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60ced4-88e8-4ffb-9ad9-d4eef42a5cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:31.663310 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871656c-f7e3-47ee-b3d4-3c2ab80ae90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:31.664044 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871656c-f7e3-47ee-b3d4-3c2ab80ae90e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:31.664394 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871656c-f7e3-47ee-b3d4-3c2ab80ae90e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:31.664601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871656c-f7e3-47ee-b3d4-3c2ab80ae90e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:31.678285 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871656c-f7e3-47ee-b3d4-3c2ab80ae90e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:31.679859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871656c-f7e3-47ee-b3d4-3c2ab80ae90e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:31.685048 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-9e60ced4-88e8-4ffb-9ad9-d4eef42a5cc8 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:31.688244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9e60ced4-88e8-4ffb-9ad9-d4eef42a5cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-a871656c-f7e3-47ee-b3d4-3c2ab80ae90e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:42:31.698011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:31.699076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:31.700751 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:42:31.700751 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:42:31.700751 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:31.700751 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:31.700751 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 521/1030] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:31.708410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871656c-f7e3-47ee-b3d4-3c2ab80ae90e None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:31.815942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:31.827530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 510/1031] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:42:31.849054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871656c-f7e3-47ee-b3d4-3c2ab80ae90e None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:31.850659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60ced4-88e8-4ffb-9ad9-d4eef42a5cc8 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:31.885185 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:31.914257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f5e3a-9105-4d7d-9fce-7c1f2421ddbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:31.914894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f5e3a-9105-4d7d-9fce-7c1f2421ddbb None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:31.915184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f5e3a-9105-4d7d-9fce-7c1f2421ddbb None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:31.915456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f5e3a-9105-4d7d-9fce-7c1f2421ddbb None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:31.916847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f5e3a-9105-4d7d-9fce-7c1f2421ddbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:31.917161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f5e3a-9105-4d7d-9fce-7c1f2421ddbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:31.919346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f5e3a-9105-4d7d-9fce-7c1f2421ddbb None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:31.941101 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 522/1032] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:31.949361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f5e3a-9105-4d7d-9fce-7c1f2421ddbb None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:32.013965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60ced4-88e8-4ffb-9ad9-d4eef42a5cc8 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:32.063152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd302b-8664-40d3-9ad1-39b32dbeb4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:32.063972 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd302b-8664-40d3-9ad1-39b32dbeb4c2 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:32.064256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd302b-8664-40d3-9ad1-39b32dbeb4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:32.064540 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd302b-8664-40d3-9ad1-39b32dbeb4c2 None admin] PATH_INFO: `/v3/services/image` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:32.065196 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd302b-8664-40d3-9ad1-39b32dbeb4c2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:32.066597 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd302b-8664-40d3-9ad1-39b32dbeb4c2 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-c5cd302b-8664-40d3-9ad1-39b32dbeb4c2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:32.079337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:32.081534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:32.083287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 511/1033] 10.222.0.94 () {56 vars in 1105 bytes} [Tue Jul 26 14:42:31 2022] GET /identity/v3/services/image => generated 86 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:32.091188 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd302b-8664-40d3-9ad1-39b32dbeb4c2 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:32.127954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f5e3a-9105-4d7d-9fce-7c1f2421ddbb None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:32.180798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44b547-0b36-4a59-9214-40d58ca112f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:32.181710 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44b547-0b36-4a59-9214-40d58ca112f8 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:32.182131 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44b547-0b36-4a59-9214-40d58ca112f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:32.182555 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44b547-0b36-4a59-9214-40d58ca112f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:32.187965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:32.195296 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44b547-0b36-4a59-9214-40d58ca112f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:32.197133 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44b547-0b36-4a59-9214-40d58ca112f8 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-fc44b547-0b36-4a59-9214-40d58ca112f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:42:32.214380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:32.216388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:32.218349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:32.218349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:32.218349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:42:32.218349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:42:32.218349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:42:32.218349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:42:32.218349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:42:32.218349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:42:32.218349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:42:32.218349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:32.218349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 523/1034] 10.222.0.94 () {56 vars in 1093 bytes} [Tue Jul 26 14:42:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:32.224925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44b547-0b36-4a59-9214-40d58ca112f8 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:32.293331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:32.313251 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd302b-8664-40d3-9ad1-39b32dbeb4c2 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:32.374590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63620c-02e4-4479-89e6-6e695a8c2aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:32.375648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63620c-02e4-4479-89e6-6e695a8c2aea None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:32.375982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63620c-02e4-4479-89e6-6e695a8c2aea None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:32.376263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63620c-02e4-4479-89e6-6e695a8c2aea None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:32.377200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63620c-02e4-4479-89e6-6e695a8c2aea None admin] RBAC: Adding query filter params (name=image) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:32.377511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63620c-02e4-4479-89e6-6e695a8c2aea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:32.379492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63620c-02e4-4479-89e6-6e695a8c2aea None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:32.385586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 512/1035] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:32.397935 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63620c-02e4-4479-89e6-6e695a8c2aea None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:32.438651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44b547-0b36-4a59-9214-40d58ca112f8 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:32.501140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7794-6e6f-487b-a250-cc4b2ec2a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:32.501773 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7794-6e6f-487b-a250-cc4b2ec2a667 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:32.502112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7794-6e6f-487b-a250-cc4b2ec2a667 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:32.502390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7794-6e6f-487b-a250-cc4b2ec2a667 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:32.503204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f7794-6e6f-487b-a250-cc4b2ec2a667 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:32.503550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f7794-6e6f-487b-a250-cc4b2ec2a667 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:32.505049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f7794-6e6f-487b-a250-cc4b2ec2a667 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:32.526773 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 524/1036] 10.222.0.94 () {56 vars in 1133 bytes} [Tue Jul 26 14:42:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:32.533956 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7794-6e6f-487b-a250-cc4b2ec2a667 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:32.600057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63620c-02e4-4479-89e6-6e695a8c2aea None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:32.606708 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:32.672409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7794-6e6f-487b-a250-cc4b2ec2a667 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:32.682054 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbb877-d3e2-46ab-abe5-de781d9060ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:32.683310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbb877-d3e2-46ab-abe5-de781d9060ad None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:32.683864 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbb877-d3e2-46ab-abe5-de781d9060ad None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:32.684352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbb877-d3e2-46ab-abe5-de781d9060ad None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:32.685635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cbb877-d3e2-46ab-abe5-de781d9060ad None admin] RBAC: Adding query filter params (type=image) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:32.686900 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cbb877-d3e2-46ab-abe5-de781d9060ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:32.689598 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cbb877-d3e2-46ab-abe5-de781d9060ad None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:32.704619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 513/1037] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:32 2022] GET /identity/v3/services?type=image => generated 350 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:32.706339 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbb877-d3e2-46ab-abe5-de781d9060ad None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:32.731303 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:32.735257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bb5de-094c-4632-9741-f2fdb7c6e249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:32.736389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bb5de-094c-4632-9741-f2fdb7c6e249 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:32.743585 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bb5de-094c-4632-9741-f2fdb7c6e249 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:32.743585 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bb5de-094c-4632-9741-f2fdb7c6e249 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:32.839159 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bb5de-094c-4632-9741-f2fdb7c6e249 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:32.840695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bb5de-094c-4632-9741-f2fdb7c6e249 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:32.885044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbb877-d3e2-46ab-abe5-de781d9060ad None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:32.932770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:32.949047 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485cf77-9f2e-4606-8ab4-9fe6fec84687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:32.949047 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485cf77-9f2e-4606-8ab4-9fe6fec84687 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:32.949047 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485cf77-9f2e-4606-8ab4-9fe6fec84687 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:32.949047 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485cf77-9f2e-4606-8ab4-9fe6fec84687 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:32.949704 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485cf77-9f2e-4606-8ab4-9fe6fec84687 None admin] RBAC: Adding query filter params (interface=public, service_id=16282c5b4acb40ba9b62def797f94429, region_id=RegionOne) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:32.949704 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485cf77-9f2e-4606-8ab4-9fe6fec84687 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:32.952850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485cf77-9f2e-4606-8ab4-9fe6fec84687 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:32.963885 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 514/1038] 10.222.0.94 () {56 vars in 1251 bytes} [Tue Jul 26 14:42:32 2022] GET /identity/v3/endpoints?service_id=16282c5b4acb40ba9b62def797f94429&interface=public®ion_id=RegionOne => generated 194 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:32.973482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485cf77-9f2e-4606-8ab4-9fe6fec84687 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:33.024587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 525/1039] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:42:32 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 14:42:33.026159 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bb5de-094c-4632-9741-f2fdb7c6e249 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:33.136198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:33.170191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485cf77-9f2e-4606-8ab4-9fe6fec84687 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:33.229515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bb5de-094c-4632-9741-f2fdb7c6e249 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:33.235119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99adef-0f28-40ef-8c1f-47b377cc87ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:33.235811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99adef-0f28-40ef-8c1f-47b377cc87ba None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:33.236283 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99adef-0f28-40ef-8c1f-47b377cc87ba None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:33.236558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99adef-0f28-40ef-8c1f-47b377cc87ba None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:33.237296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99adef-0f28-40ef-8c1f-47b377cc87ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:33.238778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99adef-0f28-40ef-8c1f-47b377cc87ba None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:33.255190 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 515/1040] 10.222.0.94 () {56 vars in 1087 bytes} [Tue Jul 26 14:42:32 2022] GET /identity/v3/services => generated 1957 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:42:33.307346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca89ca-68fc-4edc-8e17-d9c2cd00b33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:33.308571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca89ca-68fc-4edc-8e17-d9c2cd00b33a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:33.309080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca89ca-68fc-4edc-8e17-d9c2cd00b33a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:33.309582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca89ca-68fc-4edc-8e17-d9c2cd00b33a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:33.326263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca89ca-68fc-4edc-8e17-d9c2cd00b33a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:33.328332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca89ca-68fc-4edc-8e17-d9c2cd00b33a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-95ca89ca-68fc-4edc-8e17-d9c2cd00b33a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:42:33.350333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:33.360422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:33.362267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:42:33.362267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:42:33.362267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:33.362267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:33.362267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 526/1041] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:33.362769 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99adef-0f28-40ef-8c1f-47b377cc87ba None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:33.523523 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99adef-0f28-40ef-8c1f-47b377cc87ba None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:33.575084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24560424-1a86-4ff4-b484-f44d06e3a461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:33.575774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24560424-1a86-4ff4-b484-f44d06e3a461 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:33.576054 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24560424-1a86-4ff4-b484-f44d06e3a461 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:33.576330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24560424-1a86-4ff4-b484-f44d06e3a461 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:33.577755 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24560424-1a86-4ff4-b484-f44d06e3a461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:33.578300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24560424-1a86-4ff4-b484-f44d06e3a461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:33.579574 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24560424-1a86-4ff4-b484-f44d06e3a461 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:33.597822 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 516/1042] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:33.607064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca89ca-68fc-4edc-8e17-d9c2cd00b33a None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:33.620313 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:33.641284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af71eb5-891a-4c49-a209-59569d30d6d1 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:33.641824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af71eb5-891a-4c49-a209-59569d30d6d1 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:33.642321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af71eb5-891a-4c49-a209-59569d30d6d1 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:33.644992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 517/1043] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:42:33.655183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5870e0a-3aa9-4170-af72-dc9351f29ede None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:33.655811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5870e0a-3aa9-4170-af72-dc9351f29ede None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:33.656237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5870e0a-3aa9-4170-af72-dc9351f29ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:33.680168 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:33.689541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-c5870e0a-3aa9-4170-af72-dc9351f29ede None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:33.691885 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c5870e0a-3aa9-4170-af72-dc9351f29ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:33.766717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca89ca-68fc-4edc-8e17-d9c2cd00b33a None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:33.821131 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243865db-2f5c-4315-8265-4835b983b5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:33.822085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243865db-2f5c-4315-8265-4835b983b5b0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:33.822555 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243865db-2f5c-4315-8265-4835b983b5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:33.823043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243865db-2f5c-4315-8265-4835b983b5b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:33.835564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 518/1044] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:33.840469 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243865db-2f5c-4315-8265-4835b983b5b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:33.842881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243865db-2f5c-4315-8265-4835b983b5b0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-243865db-2f5c-4315-8265-4835b983b5b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:42:33.861589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:33.863727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:33.864933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:42:33.864933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:42:33.864933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:42:33.864933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:42:33.864933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:42:33.864933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:42:33.864933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:33.864933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:33.864933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 527/1045] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:33.867159 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5870e0a-3aa9-4170-af72-dc9351f29ede None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:33.875778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243865db-2f5c-4315-8265-4835b983b5b0 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:33.996539 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:34.053198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243865db-2f5c-4315-8265-4835b983b5b0 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:34.067193 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5870e0a-3aa9-4170-af72-dc9351f29ede None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:34.123186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c75aa-440c-48dd-8ebb-0388d9f1d394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:34.125024 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236507e-aadc-4b90-b23a-6d85b1c8676c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:34.125428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c75aa-440c-48dd-8ebb-0388d9f1d394 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:34.125902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236507e-aadc-4b90-b23a-6d85b1c8676c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:34.126061 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c75aa-440c-48dd-8ebb-0388d9f1d394 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:34.126330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236507e-aadc-4b90-b23a-6d85b1c8676c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:34.126491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c75aa-440c-48dd-8ebb-0388d9f1d394 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:34.126637 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236507e-aadc-4b90-b23a-6d85b1c8676c None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:34.127580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236507e-aadc-4b90-b23a-6d85b1c8676c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:34.127922 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236507e-aadc-4b90-b23a-6d85b1c8676c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:34.129525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236507e-aadc-4b90-b23a-6d85b1c8676c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:34.132917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:34.155771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 528/1046] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:42:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:34.156016 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c75aa-440c-48dd-8ebb-0388d9f1d394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:42:34.159002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236507e-aadc-4b90-b23a-6d85b1c8676c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-790c75aa-440c-48dd-8ebb-0388d9f1d394 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:42:34.163446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:34.164680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:34.165676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:34.165676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:34.165676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:34.165676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:34.165676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:42:34.165676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:42:34.165676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:42:34.165676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:42:34.165676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:42:34.165676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:42:34.165676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:42:34.165676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:34.167742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 519/1047] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:42:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:34.178713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c75aa-440c-48dd-8ebb-0388d9f1d394 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:34.349080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236507e-aadc-4b90-b23a-6d85b1c8676c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:34.377259 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c75aa-440c-48dd-8ebb-0388d9f1d394 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:34.388431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:34.400074 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120eea33-06f5-4f34-b378-24841655e062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:34.405094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120eea33-06f5-4f34-b378-24841655e062 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:34.405094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120eea33-06f5-4f34-b378-24841655e062 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:34.405094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120eea33-06f5-4f34-b378-24841655e062 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:34.405094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120eea33-06f5-4f34-b378-24841655e062 None admin] RBAC: Adding query filter params (role.id=57e2f79cce1a4685848082d34f1eb041, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=6a4278fe995e432d88f4346d6f54218b) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:34.405094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120eea33-06f5-4f34-b378-24841655e062 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:34.405094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120eea33-06f5-4f34-b378-24841655e062 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:34.412895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 529/1048] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:42:34 2022] GET /identity/v3/role_assignments?user.id=6a4278fe995e432d88f4346d6f54218b&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=57e2f79cce1a4685848082d34f1eb041 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:34.450795 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326be9da-e95b-4c7b-a63f-0bd2d33d1949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:34.452382 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326be9da-e95b-4c7b-a63f-0bd2d33d1949 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:34.452788 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326be9da-e95b-4c7b-a63f-0bd2d33d1949 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:34.453198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326be9da-e95b-4c7b-a63f-0bd2d33d1949 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:34.454333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326be9da-e95b-4c7b-a63f-0bd2d33d1949 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:34.454785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326be9da-e95b-4c7b-a63f-0bd2d33d1949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:34.457291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326be9da-e95b-4c7b-a63f-0bd2d33d1949 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:34.483529 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 520/1049] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:42:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:34.487311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120eea33-06f5-4f34-b378-24841655e062 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:34.674193 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120eea33-06f5-4f34-b378-24841655e062 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:34.722112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ce128-c265-4ef6-910e-0e7228923c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:34.723093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ce128-c265-4ef6-910e-0e7228923c01 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:34.723093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ce128-c265-4ef6-910e-0e7228923c01 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:34.723369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ce128-c265-4ef6-910e-0e7228923c01 None admin] PATH_INFO: `/v3/users/adc94d35bbdb4c8d839069c7b8235f5b` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:34.740858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ce128-c265-4ef6-910e-0e7228923c01 None admin] RBAC: Authorizing `identity:get_user(user_id=adc94d35bbdb4c8d839069c7b8235f5b)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:34.742331 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ce128-c265-4ef6-910e-0e7228923c01 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:34.744420 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 530/1050] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:42:34 2022] GET /identity/v3/users/adc94d35bbdb4c8d839069c7b8235f5b => generated 285 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:34.751221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326be9da-e95b-4c7b-a63f-0bd2d33d1949 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:34.836316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:34.899891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326be9da-e95b-4c7b-a63f-0bd2d33d1949 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:34.948003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ddabc-3f41-4d24-a306-2469fb4b9953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:34.948596 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ddabc-3f41-4d24-a306-2469fb4b9953 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:34.948877 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ddabc-3f41-4d24-a306-2469fb4b9953 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:34.949228 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ddabc-3f41-4d24-a306-2469fb4b9953 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:35.052837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ddabc-3f41-4d24-a306-2469fb4b9953 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:35.054410 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ddabc-3f41-4d24-a306-2469fb4b9953 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:35.136136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:35.207624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 521/1051] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:42:34 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:42:35.212264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ce128-c265-4ef6-910e-0e7228923c01 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:35.301710 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:35.410090 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ce128-c265-4ef6-910e-0e7228923c01 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:35.481580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e250e8-48e3-4848-ae26-ef7257677d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:35.483525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e250e8-48e3-4848-ae26-ef7257677d79 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:35.484114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e250e8-48e3-4848-ae26-ef7257677d79 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:35.484521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e250e8-48e3-4848-ae26-ef7257677d79 None admin] PATH_INFO: `/v3/projects/be587c83303a4dcc9b6deb622caf11da` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:35.486853 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e250e8-48e3-4848-ae26-ef7257677d79 None admin] RBAC: Authorizing `identity:get_project(project_id=be587c83303a4dcc9b6deb622caf11da)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:35.489590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e250e8-48e3-4848-ae26-ef7257677d79 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:35.498913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 531/1052] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:42:35 2022] GET /identity/v3/projects/be587c83303a4dcc9b6deb622caf11da => generated 311 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:35.500137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ddabc-3f41-4d24-a306-2469fb4b9953 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:35.625551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ddabc-3f41-4d24-a306-2469fb4b9953 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:35.668989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44d53e-088d-45b7-9892-927c0d0cb61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:35.669760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44d53e-088d-45b7-9892-927c0d0cb61e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:35.670072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44d53e-088d-45b7-9892-927c0d0cb61e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:35.670940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44d53e-088d-45b7-9892-927c0d0cb61e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:35.671862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44d53e-088d-45b7-9892-927c0d0cb61e None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=be587c83303a4dcc9b6deb622caf11da, user.id=adc94d35bbdb4c8d839069c7b8235f5b) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:35.672153 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44d53e-088d-45b7-9892-927c0d0cb61e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:35.673696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44d53e-088d-45b7-9892-927c0d0cb61e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:35.687513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 522/1053] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:42:35 2022] GET /identity/v3/role_assignments?user.id=adc94d35bbdb4c8d839069c7b8235f5b&scope.project.id=be587c83303a4dcc9b6deb622caf11da&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:35.776134 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be8628-da51-4b1c-9e58-64654fe3201f None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:35.776500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be8628-da51-4b1c-9e58-64654fe3201f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:35.776728 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be8628-da51-4b1c-9e58-64654fe3201f None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:35.778218 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 532/1054] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:35.785743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c36ca-0db7-48a6-8af9-047df005c7e8 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:35.786096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c36ca-0db7-48a6-8af9-047df005c7e8 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:35.786374 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c36ca-0db7-48a6-8af9-047df005c7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:35.808280 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ae8c36ca-0db7-48a6-8af9-047df005c7e8 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:35.809694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ae8c36ca-0db7-48a6-8af9-047df005c7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:35.832876 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:35.898736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 523/1055] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:35.924668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be8628-da51-4b1c-9e58-64654fe3201f None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:36.063574 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be8628-da51-4b1c-9e58-64654fe3201f None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:36.098316 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:36.127481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7e9bd-eec6-4f08-b307-0c727171485c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:36.128560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7e9bd-eec6-4f08-b307-0c727171485c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:36.129034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7e9bd-eec6-4f08-b307-0c727171485c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:36.129514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7e9bd-eec6-4f08-b307-0c727171485c None admin] PATH_INFO: `/v3/services/image` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:36.130709 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7e9bd-eec6-4f08-b307-0c727171485c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:36.133259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7e9bd-eec6-4f08-b307-0c727171485c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-6ea7e9bd-eec6-4f08-b307-0c727171485c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:36.150759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:36.152145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:36.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:36.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:36.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:36.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:36.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:36.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:36.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:36.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:36.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:36.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:36.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:42:36.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:42:36.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 14:42:36.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:36.153918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 533/1056] 10.222.0.94 () {56 vars in 1105 bytes} [Tue Jul 26 14:42:35 2022] GET /identity/v3/services/image => generated 86 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:36.162405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c36ca-0db7-48a6-8af9-047df005c7e8 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:36.375706 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c36ca-0db7-48a6-8af9-047df005c7e8 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:36.447065 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b95fb9-5477-4d18-815a-6a6ccdf9b1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:36.448143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b95fb9-5477-4d18-815a-6a6ccdf9b1b5 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:36.448647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b95fb9-5477-4d18-815a-6a6ccdf9b1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:36.449224 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b95fb9-5477-4d18-815a-6a6ccdf9b1b5 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:36.450524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b95fb9-5477-4d18-815a-6a6ccdf9b1b5 None admin] RBAC: Adding query filter params (name=image) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:36.451096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b95fb9-5477-4d18-815a-6a6ccdf9b1b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:36.453756 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b95fb9-5477-4d18-815a-6a6ccdf9b1b5 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:36.470465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 524/1057] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:36.472305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7e9bd-eec6-4f08-b307-0c727171485c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:36.675796 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7e9bd-eec6-4f08-b307-0c727171485c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:36.740202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5549ab4-96a5-45aa-b2f8-1435047d66cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:36.741190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5549ab4-96a5-45aa-b2f8-1435047d66cb None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:36.741677 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5549ab4-96a5-45aa-b2f8-1435047d66cb None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:36.742166 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5549ab4-96a5-45aa-b2f8-1435047d66cb None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:36.743396 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5549ab4-96a5-45aa-b2f8-1435047d66cb None admin] RBAC: Adding query filter params (type=image) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:36.743935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5549ab4-96a5-45aa-b2f8-1435047d66cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:36.746436 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5549ab4-96a5-45aa-b2f8-1435047d66cb None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:36.756746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 534/1058] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:36.766804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b95fb9-5477-4d18-815a-6a6ccdf9b1b5 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:36.805884 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:36.948467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b95fb9-5477-4d18-815a-6a6ccdf9b1b5 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:36.996049 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb54f81-b203-4bc3-9998-89f42e7fdec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:36.996878 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb54f81-b203-4bc3-9998-89f42e7fdec0 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:36.997316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb54f81-b203-4bc3-9998-89f42e7fdec0 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:36.997750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb54f81-b203-4bc3-9998-89f42e7fdec0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:36.999846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb54f81-b203-4bc3-9998-89f42e7fdec0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:37.001576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb54f81-b203-4bc3-9998-89f42e7fdec0 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:37.103378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 525/1059] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 14:42:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:42:37.210474 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fc1c4-d107-4640-9ea3-ed4e526c6a55 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:37.210870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fc1c4-d107-4640-9ea3-ed4e526c6a55 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:37.211326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fc1c4-d107-4640-9ea3-ed4e526c6a55 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:37.213915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 535/1060] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:37.221053 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee55aab-4cfa-433f-80d2-3375d7d6faed None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:37.221782 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee55aab-4cfa-433f-80d2-3375d7d6faed None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:37.221782 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee55aab-4cfa-433f-80d2-3375d7d6faed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:37.247289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-1ee55aab-4cfa-433f-80d2-3375d7d6faed None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:37.247289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1ee55aab-4cfa-433f-80d2-3375d7d6faed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:37.343877 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:37.353467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 526/1061] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:42:37.377936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fc1c4-d107-4640-9ea3-ed4e526c6a55 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:37.451723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:37.539768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fc1c4-d107-4640-9ea3-ed4e526c6a55 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:37.610273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfffc30-63a3-4e63-aa26-ac1b929c7ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:37.611394 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfffc30-63a3-4e63-aa26-ac1b929c7ef8 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:37.611900 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfffc30-63a3-4e63-aa26-ac1b929c7ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:37.612367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfffc30-63a3-4e63-aa26-ac1b929c7ef8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:37.634485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfffc30-63a3-4e63-aa26-ac1b929c7ef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-edfffc30-63a3-4e63-aa26-ac1b929c7ef8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:42:37.645452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:37.652289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:37.654495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:37.654495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:37.654495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:37.654495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:37.654495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:42:37.654495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:42:37.654495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:42:37.654495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:42:37.654495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:42:37.654495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:42:37.654495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:42:37.654495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:37.654495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 536/1062] 10.222.0.94 () {56 vars in 1120 bytes} [Tue Jul 26 14:42:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:37.664256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee55aab-4cfa-433f-80d2-3375d7d6faed None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:37.851681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee55aab-4cfa-433f-80d2-3375d7d6faed None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:37.899394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c2674-1a17-4d0c-8627-5765cac93645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:37.900138 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c2674-1a17-4d0c-8627-5765cac93645 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:37.900485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c2674-1a17-4d0c-8627-5765cac93645 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:37.900782 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c2674-1a17-4d0c-8627-5765cac93645 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:37.901632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c2674-1a17-4d0c-8627-5765cac93645 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:37.902019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c2674-1a17-4d0c-8627-5765cac93645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:37.903594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c2674-1a17-4d0c-8627-5765cac93645 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:37.917334 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 527/1063] 10.222.0.94 () {56 vars in 1115 bytes} [Tue Jul 26 14:42:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:37.918875 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27f16b-7936-4da9-9049-1c462ccf6b25 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:37.919165 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27f16b-7936-4da9-9049-1c462ccf6b25 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:37.919448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27f16b-7936-4da9-9049-1c462ccf6b25 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:37.922285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c2674-1a17-4d0c-8627-5765cac93645 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:37.923578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 537/1064] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:42:37.927202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba042fc-870f-4dd9-baa8-5c02150d6d08 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:37.927543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba042fc-870f-4dd9-baa8-5c02150d6d08 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:37.927916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba042fc-870f-4dd9-baa8-5c02150d6d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:37.948518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7ba042fc-870f-4dd9-baa8-5c02150d6d08 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:37.949834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7ba042fc-870f-4dd9-baa8-5c02150d6d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:38.038515 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:38.063510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 538/1065] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:42:38.082146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba042fc-870f-4dd9-baa8-5c02150d6d08 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:38.112258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c2674-1a17-4d0c-8627-5765cac93645 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:38.130386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:38.162451 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6676b-0fda-4044-b4fd-0c81270de6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:38.163413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6676b-0fda-4044-b4fd-0c81270de6f0 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:38.163413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6676b-0fda-4044-b4fd-0c81270de6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:38.163729 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6676b-0fda-4044-b4fd-0c81270de6f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:38.187840 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6676b-0fda-4044-b4fd-0c81270de6f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:38.190743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6676b-0fda-4044-b4fd-0c81270de6f0 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-d1f6676b-0fda-4044-b4fd-0c81270de6f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:42:38.219685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:38.222420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:38.225590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:42:38.225590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:42:38.225590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:38.225590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:38.225590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 528/1066] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:38.228967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6676b-0fda-4044-b4fd-0c81270de6f0 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:38.245547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba042fc-870f-4dd9-baa8-5c02150d6d08 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:38.310965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e096035-b141-4dc7-9ced-710a1f440581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:38.312380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e096035-b141-4dc7-9ced-710a1f440581 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:38.312592 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e096035-b141-4dc7-9ced-710a1f440581 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:38.312932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e096035-b141-4dc7-9ced-710a1f440581 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:38.329894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e096035-b141-4dc7-9ced-710a1f440581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-1e096035-b141-4dc7-9ced-710a1f440581 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:42:38.335863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:38.337798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:38.340389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:38.340389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:38.340389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:38.340389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:38.340389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:42:38.340389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:42:38.340389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:42:38.340389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:42:38.340389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:42:38.340389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:42:38.340389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:42:38.340389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:38.340389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 539/1067] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:42:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:38.348642 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e096035-b141-4dc7-9ced-710a1f440581 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:38.445843 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6676b-0fda-4044-b4fd-0c81270de6f0 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:38.465324 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:38.504317 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48936ade-e453-4790-b6d3-2792271060a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:38.505328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48936ade-e453-4790-b6d3-2792271060a3 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:38.505899 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48936ade-e453-4790-b6d3-2792271060a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:38.506387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48936ade-e453-4790-b6d3-2792271060a3 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:38.507784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48936ade-e453-4790-b6d3-2792271060a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:38.508314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48936ade-e453-4790-b6d3-2792271060a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:38.510929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48936ade-e453-4790-b6d3-2792271060a3 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:38.526762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:38.544372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 529/1068] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:38.544684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e096035-b141-4dc7-9ced-710a1f440581 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:38.546431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48936ade-e453-4790-b6d3-2792271060a3 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:38.624231 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489da6b1-417e-4e22-a81b-9250d8a6d1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:38.625200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489da6b1-417e-4e22-a81b-9250d8a6d1a0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:38.625657 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489da6b1-417e-4e22-a81b-9250d8a6d1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:38.626220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489da6b1-417e-4e22-a81b-9250d8a6d1a0 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:38.627598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489da6b1-417e-4e22-a81b-9250d8a6d1a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:38.628104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489da6b1-417e-4e22-a81b-9250d8a6d1a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:38.630950 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489da6b1-417e-4e22-a81b-9250d8a6d1a0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:38.655151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 540/1069] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:42:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:38.656822 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489da6b1-417e-4e22-a81b-9250d8a6d1a0 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:38.725199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48936ade-e453-4790-b6d3-2792271060a3 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:38.786371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32838165-aec5-40f5-ad0c-667c9f391921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:38.787401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32838165-aec5-40f5-ad0c-667c9f391921 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:38.787854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32838165-aec5-40f5-ad0c-667c9f391921 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:38.788262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32838165-aec5-40f5-ad0c-667c9f391921 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:38.799616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32838165-aec5-40f5-ad0c-667c9f391921 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:38.801674 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32838165-aec5-40f5-ad0c-667c9f391921 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-32838165-aec5-40f5-ad0c-667c9f391921 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:42:38.818227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:38.820237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:38.822183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:38.822183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:38.822183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:42:38.822183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:42:38.822183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:42:38.822183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:42:38.822183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:42:38.822183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:42:38.822183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:42:38.822183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:38.825057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 530/1070] 10.222.0.94 () {56 vars in 1093 bytes} [Tue Jul 26 14:42:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:38.826261 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32838165-aec5-40f5-ad0c-667c9f391921 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:38.858593 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489da6b1-417e-4e22-a81b-9250d8a6d1a0 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:38.888130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:38.907943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8148c-223a-4176-aefb-c18e3dadd34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:38.908927 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8148c-223a-4176-aefb-c18e3dadd34d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:38.909228 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8148c-223a-4176-aefb-c18e3dadd34d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:38.909536 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8148c-223a-4176-aefb-c18e3dadd34d None admin] PATH_INFO: `/v3/users/adc94d35bbdb4c8d839069c7b8235f5b` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:38.910943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8148c-223a-4176-aefb-c18e3dadd34d None admin] RBAC: Authorizing `identity:get_user(user_id=adc94d35bbdb4c8d839069c7b8235f5b)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:38.912426 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8148c-223a-4176-aefb-c18e3dadd34d None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:38.914649 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 541/1071] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:42:38 2022] GET /identity/v3/users/adc94d35bbdb4c8d839069c7b8235f5b => generated 285 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:38.921382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8148c-223a-4176-aefb-c18e3dadd34d None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:38.970615 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:39.028821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32838165-aec5-40f5-ad0c-667c9f391921 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:39.096008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8148c-223a-4176-aefb-c18e3dadd34d None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:39.100969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e7af9-64fb-42aa-a9b6-6ec711209486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:39.101963 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e7af9-64fb-42aa-a9b6-6ec711209486 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:39.102454 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e7af9-64fb-42aa-a9b6-6ec711209486 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:39.102936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e7af9-64fb-42aa-a9b6-6ec711209486 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:39.104526 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e7af9-64fb-42aa-a9b6-6ec711209486 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:39.105059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e7af9-64fb-42aa-a9b6-6ec711209486 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:39.108464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e7af9-64fb-42aa-a9b6-6ec711209486 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:39.151534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 531/1072] 10.222.0.94 () {56 vars in 1133 bytes} [Tue Jul 26 14:42:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:39.151829 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292f32d-5295-4708-b488-e53a24d43604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:39.156365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292f32d-5295-4708-b488-e53a24d43604 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:39.156365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292f32d-5295-4708-b488-e53a24d43604 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:39.156365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292f32d-5295-4708-b488-e53a24d43604 None admin] PATH_INFO: `/v3/projects/be587c83303a4dcc9b6deb622caf11da` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:39.156365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292f32d-5295-4708-b488-e53a24d43604 None admin] RBAC: Authorizing `identity:get_project(project_id=be587c83303a4dcc9b6deb622caf11da)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:39.157319 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292f32d-5295-4708-b488-e53a24d43604 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:39.158085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e7af9-64fb-42aa-a9b6-6ec711209486 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:39.159804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 542/1073] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:42:38 2022] GET /identity/v3/projects/be587c83303a4dcc9b6deb622caf11da => generated 311 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:39.172725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292f32d-5295-4708-b488-e53a24d43604 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:39.332526 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:39.352001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e7af9-64fb-42aa-a9b6-6ec711209486 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:39.370218 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:39.387008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292f32d-5295-4708-b488-e53a24d43604 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:39.421576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170ff15-f420-487e-9623-cf080cab9371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:39.422777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170ff15-f420-487e-9623-cf080cab9371 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:39.423277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170ff15-f420-487e-9623-cf080cab9371 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:39.424369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170ff15-f420-487e-9623-cf080cab9371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:39.446980 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170ff15-f420-487e-9623-cf080cab9371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:39.449954 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170ff15-f420-487e-9623-cf080cab9371 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:39.461391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8cfa8-fe8c-4df1-91d4-fb4cac3984e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:39.462221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8cfa8-fe8c-4df1-91d4-fb4cac3984e4 None admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:39.462658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8cfa8-fe8c-4df1-91d4-fb4cac3984e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:39.462935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8cfa8-fe8c-4df1-91d4-fb4cac3984e4 None admin] PATH_INFO: `/v3/projects/be587c83303a4dcc9b6deb622caf11da/users/adc94d35bbdb4c8d839069c7b8235f5b/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:39.465368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8cfa8-fe8c-4df1-91d4-fb4cac3984e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=be587c83303a4dcc9b6deb622caf11da, user_id=adc94d35bbdb4c8d839069c7b8235f5b, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:39.467798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8cfa8-fe8c-4df1-91d4-fb4cac3984e4 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-0170ff15-f420-487e-9623-cf080cab9371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:42:39.483928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:39.485275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:39.486394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:42:39.486394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:42:39.486394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:39.486394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:39.486394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 532/1074] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:39.495686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170ff15-f420-487e-9623-cf080cab9371 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:39.579205 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 543/1075] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:42:39 2022] PUT /identity/v3/projects/be587c83303a4dcc9b6deb622caf11da/users/adc94d35bbdb4c8d839069c7b8235f5b/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:42:39.682926 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170ff15-f420-487e-9623-cf080cab9371 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:39.734644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe1805-865f-4f1c-801c-8ad1f592d054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:39.735407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe1805-865f-4f1c-801c-8ad1f592d054 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:39.735732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe1805-865f-4f1c-801c-8ad1f592d054 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:39.736015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe1805-865f-4f1c-801c-8ad1f592d054 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:39.736774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe1805-865f-4f1c-801c-8ad1f592d054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:39.737089 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe1805-865f-4f1c-801c-8ad1f592d054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:39.738668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe1805-865f-4f1c-801c-8ad1f592d054 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:39.748421 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbda81-8ab1-42f4-a4c4-e389a7901447 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:39.748794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbda81-8ab1-42f4-a4c4-e389a7901447 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:39.749104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbda81-8ab1-42f4-a4c4-e389a7901447 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:39.750901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 544/1076] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:42:39.757404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 533/1077] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:39.757682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da7cb5-5997-4076-b11c-c11a4868ef4b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:39.758087 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da7cb5-5997-4076-b11c-c11a4868ef4b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:39.758637 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da7cb5-5997-4076-b11c-c11a4868ef4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:39.767082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe1805-865f-4f1c-801c-8ad1f592d054 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:39.829344 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:39.829583 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-e8da7cb5-5997-4076-b11c-c11a4868ef4b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:39.829583 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e8da7cb5-5997-4076-b11c-c11a4868ef4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:39.906488 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 545/1078] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:42:39.929944 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da7cb5-5997-4076-b11c-c11a4868ef4b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:39.946934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe1805-865f-4f1c-801c-8ad1f592d054 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:40.000916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89fe54-0796-4c14-b733-7dc4c840192d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:40.002198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89fe54-0796-4c14-b733-7dc4c840192d None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:40.002720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89fe54-0796-4c14-b733-7dc4c840192d None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:40.003315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89fe54-0796-4c14-b733-7dc4c840192d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:40.013462 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89fe54-0796-4c14-b733-7dc4c840192d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:40.016672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89fe54-0796-4c14-b733-7dc4c840192d None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-2b89fe54-0796-4c14-b733-7dc4c840192d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:42:40.033871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:40.035119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:40.036291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:42:40.036291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:42:40.036291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:42:40.036291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:42:40.036291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:42:40.036291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:42:40.036291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:40.036291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:40.038276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 534/1079] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:42:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:40.044619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89fe54-0796-4c14-b733-7dc4c840192d None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:40.053875 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:40.081554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da7cb5-5997-4076-b11c-c11a4868ef4b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:40.137641 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b49a8-4a75-419d-8d68-9c4d15eb1787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:40.138297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b49a8-4a75-419d-8d68-9c4d15eb1787 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:40.138716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b49a8-4a75-419d-8d68-9c4d15eb1787 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:40.139682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b49a8-4a75-419d-8d68-9c4d15eb1787 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:40.161654 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:40.223226 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89fe54-0796-4c14-b733-7dc4c840192d None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:40.297605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b49a8-4a75-419d-8d68-9c4d15eb1787 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:40.300325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b49a8-4a75-419d-8d68-9c4d15eb1787 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:40.303600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de71525-65ad-434e-b533-4c83d4420164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:40.304258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de71525-65ad-434e-b533-4c83d4420164 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:40.304810 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de71525-65ad-434e-b533-4c83d4420164 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:40.306711 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de71525-65ad-434e-b533-4c83d4420164 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:40.306711 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de71525-65ad-434e-b533-4c83d4420164 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:40.306711 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de71525-65ad-434e-b533-4c83d4420164 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:40.308432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de71525-65ad-434e-b533-4c83d4420164 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:40.330985 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 535/1080] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:42:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:40.337549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de71525-65ad-434e-b533-4c83d4420164 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:40.449935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:40.485410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 546/1081] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:42:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 14:42:40.495412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b49a8-4a75-419d-8d68-9c4d15eb1787 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:40.530145 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de71525-65ad-434e-b533-4c83d4420164 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:40.578477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:40.600289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8cd0d-5ae5-427f-b746-57ac44220a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:40.601422 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8cd0d-5ae5-427f-b746-57ac44220a7b None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:40.601922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8cd0d-5ae5-427f-b746-57ac44220a7b None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:40.602434 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8cd0d-5ae5-427f-b746-57ac44220a7b None admin] PATH_INFO: `/v3/projects/a9b328041da24cedbc801b271d6e0419/users/6a4278fe995e432d88f4346d6f54218b/roles/57e2f79cce1a4685848082d34f1eb041` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:40.606641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8cd0d-5ae5-427f-b746-57ac44220a7b None admin] RBAC: Authorizing `identity:create_grant(project_id=a9b328041da24cedbc801b271d6e0419, user_id=6a4278fe995e432d88f4346d6f54218b, role_id=57e2f79cce1a4685848082d34f1eb041)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:40.609850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8cd0d-5ae5-427f-b746-57ac44220a7b None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:40.663080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b49a8-4a75-419d-8d68-9c4d15eb1787 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:40.717500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a657f0a-e549-49a2-b5a9-46a9ad5245cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:40.718648 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657f0a-e549-49a2-b5a9-46a9ad5245cd None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:40.719103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657f0a-e549-49a2-b5a9-46a9ad5245cd None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:40.719660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657f0a-e549-49a2-b5a9-46a9ad5245cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:40.740720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a657f0a-e549-49a2-b5a9-46a9ad5245cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:40.743664 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a657f0a-e549-49a2-b5a9-46a9ad5245cd None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-0a657f0a-e549-49a2-b5a9-46a9ad5245cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:42:40.775499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:40.776725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:40.777825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:42:40.777825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:42:40.777825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:40.777825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:40.784649 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 547/1082] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:40.786789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a657f0a-e549-49a2-b5a9-46a9ad5245cd None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:40.792646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 536/1083] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:42:40 2022] PUT /identity/v3/projects/a9b328041da24cedbc801b271d6e0419/users/6a4278fe995e432d88f4346d6f54218b/roles/57e2f79cce1a4685848082d34f1eb041 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 14:42:40.868353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:40.970163 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a657f0a-e549-49a2-b5a9-46a9ad5245cd None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:41.026216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82ee17-7170-403c-8c32-edd0dc9fcff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:41.026905 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82ee17-7170-403c-8c32-edd0dc9fcff5 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:41.027188 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82ee17-7170-403c-8c32-edd0dc9fcff5 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:41.027518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82ee17-7170-403c-8c32-edd0dc9fcff5 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:41.028368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82ee17-7170-403c-8c32-edd0dc9fcff5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:41.028681 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82ee17-7170-403c-8c32-edd0dc9fcff5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:41.030406 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82ee17-7170-403c-8c32-edd0dc9fcff5 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:41.049668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 548/1084] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:41.900782 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565b12f-8d7d-4f4d-b260-48746771d8b5 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:41.901432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565b12f-8d7d-4f4d-b260-48746771d8b5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:41.901432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565b12f-8d7d-4f4d-b260-48746771d8b5 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:41.902736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 537/1085] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:41.909901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e81b3d-5dc4-4c92-9433-46ee56d36b59 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:41.910573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e81b3d-5dc4-4c92-9433-46ee56d36b59 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:41.910573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e81b3d-5dc4-4c92-9433-46ee56d36b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:41.932126 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-53e81b3d-5dc4-4c92-9433-46ee56d36b59 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:41.933360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-53e81b3d-5dc4-4c92-9433-46ee56d36b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:42.019343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 549/1086] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:42.044472 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565b12f-8d7d-4f4d-b260-48746771d8b5 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:42.187897 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565b12f-8d7d-4f4d-b260-48746771d8b5 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:42.239563 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4afb8-2ff9-4f3d-a37f-4022475f3ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:42.240227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4afb8-2ff9-4f3d-a37f-4022475f3ed2 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:42.240508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4afb8-2ff9-4f3d-a37f-4022475f3ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:42.240846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4afb8-2ff9-4f3d-a37f-4022475f3ed2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:42.256390 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4afb8-2ff9-4f3d-a37f-4022475f3ed2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-1dc4afb8-2ff9-4f3d-a37f-4022475f3ed2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:42:42.261813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:42.263045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:42.264760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:42.264760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:42.264760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:42.264760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:42.264760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:42:42.264760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:42:42.264760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:42:42.264760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:42:42.264760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:42:42.264760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:42:42.264760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:42:42.264760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:42.267863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 538/1087] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:42:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:42.270448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e81b3d-5dc4-4c92-9433-46ee56d36b59 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:42.302952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:42.403321 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e81b3d-5dc4-4c92-9433-46ee56d36b59 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:42.451023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ed64e-63d5-4704-8d3d-6c9025b34518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:42.451897 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ed64e-63d5-4704-8d3d-6c9025b34518 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:42.452089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ed64e-63d5-4704-8d3d-6c9025b34518 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:42.452383 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ed64e-63d5-4704-8d3d-6c9025b34518 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:42.453190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ed64e-63d5-4704-8d3d-6c9025b34518 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:42.453547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ed64e-63d5-4704-8d3d-6c9025b34518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:42.455272 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ed64e-63d5-4704-8d3d-6c9025b34518 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:42.465903 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 550/1088] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:42:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:42.473525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4afb8-2ff9-4f3d-a37f-4022475f3ed2 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:42.552871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:42.627271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4afb8-2ff9-4f3d-a37f-4022475f3ed2 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:42.683710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d9c9d-cd2c-4a4d-a5f7-cc20f6eccac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:42.684888 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d9c9d-cd2c-4a4d-a5f7-cc20f6eccac1 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:42.685457 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d9c9d-cd2c-4a4d-a5f7-cc20f6eccac1 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:42.685953 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d9c9d-cd2c-4a4d-a5f7-cc20f6eccac1 None admin] PATH_INFO: `/v3/users/adc94d35bbdb4c8d839069c7b8235f5b` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:42.688478 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d9c9d-cd2c-4a4d-a5f7-cc20f6eccac1 None admin] RBAC: Authorizing `identity:get_user(user_id=adc94d35bbdb4c8d839069c7b8235f5b)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:42.690935 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d9c9d-cd2c-4a4d-a5f7-cc20f6eccac1 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:42.699522 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 539/1089] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:42:42 2022] GET /identity/v3/users/adc94d35bbdb4c8d839069c7b8235f5b => generated 285 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:42.702274 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ed64e-63d5-4704-8d3d-6c9025b34518 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:42.871897 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:42.901959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ed64e-63d5-4704-8d3d-6c9025b34518 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:42.956093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade24bd-0515-40e1-9aa4-18834ac194a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:42.957056 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade24bd-0515-40e1-9aa4-18834ac194a6 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:42.957554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade24bd-0515-40e1-9aa4-18834ac194a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:42.958039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade24bd-0515-40e1-9aa4-18834ac194a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:43.076617 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade24bd-0515-40e1-9aa4-18834ac194a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:43.079226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade24bd-0515-40e1-9aa4-18834ac194a6 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:43.197237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:43.218881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 551/1090] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:42:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:42:43.225332 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d9c9d-cd2c-4a4d-a5f7-cc20f6eccac1 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:43.376400 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d9c9d-cd2c-4a4d-a5f7-cc20f6eccac1 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:43.397393 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:43.434251 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e815565-01dd-4b02-8396-be90fe850fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:43.435031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e815565-01dd-4b02-8396-be90fe850fcb None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:43.435693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e815565-01dd-4b02-8396-be90fe850fcb None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:43.436498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e815565-01dd-4b02-8396-be90fe850fcb None admin] PATH_INFO: `/v3/projects/be587c83303a4dcc9b6deb622caf11da` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:43.437994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e815565-01dd-4b02-8396-be90fe850fcb None admin] RBAC: Authorizing `identity:get_project(project_id=be587c83303a4dcc9b6deb622caf11da)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:43.439965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e815565-01dd-4b02-8396-be90fe850fcb None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:43.441922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 540/1091] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:42:43 2022] GET /identity/v3/projects/be587c83303a4dcc9b6deb622caf11da => generated 311 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:43.448463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade24bd-0515-40e1-9aa4-18834ac194a6 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:43.606387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd13f2-144a-470d-85ff-94b8674fa803 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:43.606887 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd13f2-144a-470d-85ff-94b8674fa803 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:43.607405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd13f2-144a-470d-85ff-94b8674fa803 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:43.610186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 541/1092] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:43.618599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945ad34-3e85-4b1f-9f84-bb6530d2c0e0 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:43.619056 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945ad34-3e85-4b1f-9f84-bb6530d2c0e0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:43.619259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade24bd-0515-40e1-9aa4-18834ac194a6 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:43.619493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945ad34-3e85-4b1f-9f84-bb6530d2c0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:43.648670 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-a945ad34-3e85-4b1f-9f84-bb6530d2c0e0 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:43.649936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a945ad34-3e85-4b1f-9f84-bb6530d2c0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:43.676909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4987c-e1f8-4635-9ba2-2d358baf116e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:43.677671 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4987c-e1f8-4635-9ba2-2d358baf116e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:43.677954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4987c-e1f8-4635-9ba2-2d358baf116e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:43.678229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4987c-e1f8-4635-9ba2-2d358baf116e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:43.679369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4987c-e1f8-4635-9ba2-2d358baf116e None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=be587c83303a4dcc9b6deb622caf11da, user.id=adc94d35bbdb4c8d839069c7b8235f5b) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:43.679930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4987c-e1f8-4635-9ba2-2d358baf116e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:43.681841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4987c-e1f8-4635-9ba2-2d358baf116e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:43.692860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 552/1093] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:42:43 2022] GET /identity/v3/role_assignments?user.id=adc94d35bbdb4c8d839069c7b8235f5b&scope.project.id=be587c83303a4dcc9b6deb622caf11da&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:43.777952 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 542/1094] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:42:43.814016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4987c-e1f8-4635-9ba2-2d358baf116e None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:43.912097 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:44.002280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4987c-e1f8-4635-9ba2-2d358baf116e None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:44.081032 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdcdaa-82f1-4e57-9437-7d1be707a204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:44.081929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdcdaa-82f1-4e57-9437-7d1be707a204 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:44.082299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdcdaa-82f1-4e57-9437-7d1be707a204 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:44.082739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdcdaa-82f1-4e57-9437-7d1be707a204 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:44.098142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdcdaa-82f1-4e57-9437-7d1be707a204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-83cdcdaa-82f1-4e57-9437-7d1be707a204 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:42:44.104782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:44.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:44.109377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:44.109377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:44.109377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:44.109377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:44.109377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:42:44.109377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:42:44.109377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:42:44.109377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:42:44.109377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:42:44.109377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:42:44.109377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 14:42:44.109377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:44.109377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 553/1095] 10.222.0.94 () {56 vars in 1120 bytes} [Tue Jul 26 14:42:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:44.113452 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945ad34-3e85-4b1f-9f84-bb6530d2c0e0 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:44.308538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945ad34-3e85-4b1f-9f84-bb6530d2c0e0 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:44.321473 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:44.365345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d8fe6-d0ce-471d-b63f-defc4dcaf343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:44.366429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d8fe6-d0ce-471d-b63f-defc4dcaf343 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:44.366897 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d8fe6-d0ce-471d-b63f-defc4dcaf343 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:44.367377 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d8fe6-d0ce-471d-b63f-defc4dcaf343 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:44.368768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d8fe6-d0ce-471d-b63f-defc4dcaf343 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:44.369305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d8fe6-d0ce-471d-b63f-defc4dcaf343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:44.372196 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d8fe6-d0ce-471d-b63f-defc4dcaf343 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:44.391817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 543/1096] 10.222.0.94 () {56 vars in 1115 bytes} [Tue Jul 26 14:42:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:44.401196 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdcdaa-82f1-4e57-9437-7d1be707a204 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:44.542536 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdcdaa-82f1-4e57-9437-7d1be707a204 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:44.558942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:44.590494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdecdc2-eb5f-4092-863f-9d11ee1a7317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:44.591167 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdecdc2-eb5f-4092-863f-9d11ee1a7317 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:44.592218 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdecdc2-eb5f-4092-863f-9d11ee1a7317 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:44.592539 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdecdc2-eb5f-4092-863f-9d11ee1a7317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:44.607047 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdecdc2-eb5f-4092-863f-9d11ee1a7317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:44.608943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdecdc2-eb5f-4092-863f-9d11ee1a7317 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-dcdecdc2-eb5f-4092-863f-9d11ee1a7317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:42:44.631675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:44.633579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:44.635429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:42:44.635429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:42:44.635429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:44.635429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:44.639795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 554/1097] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:44.641927 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d8fe6-d0ce-471d-b63f-defc4dcaf343 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:44.780837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d8fe6-d0ce-471d-b63f-defc4dcaf343 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:44.829720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be9475-5399-4b24-97fc-83ccf4f79cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:44.830273 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be9475-5399-4b24-97fc-83ccf4f79cf1 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:44.830652 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be9475-5399-4b24-97fc-83ccf4f79cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:44.830959 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be9475-5399-4b24-97fc-83ccf4f79cf1 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:44.831960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be9475-5399-4b24-97fc-83ccf4f79cf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:44.832368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be9475-5399-4b24-97fc-83ccf4f79cf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:44.833980 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be9475-5399-4b24-97fc-83ccf4f79cf1 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:44.858917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 544/1098] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:44.861783 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdecdc2-eb5f-4092-863f-9d11ee1a7317 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:45.003135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdecdc2-eb5f-4092-863f-9d11ee1a7317 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:45.056635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b590710-17cf-4959-ac3c-8158dfef1c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:45.057369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b590710-17cf-4959-ac3c-8158dfef1c9c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:45.057647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b590710-17cf-4959-ac3c-8158dfef1c9c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:45.058039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b590710-17cf-4959-ac3c-8158dfef1c9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:45.068220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b590710-17cf-4959-ac3c-8158dfef1c9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:45.069694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b590710-17cf-4959-ac3c-8158dfef1c9c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-4b590710-17cf-4959-ac3c-8158dfef1c9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:42:45.084534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:45.095382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:45.097700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:45.097700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:45.097700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:42:45.097700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:42:45.097700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:42:45.097700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:42:45.097700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:42:45.097700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:42:45.097700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 14:42:45.097700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:45.097700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 555/1099] 10.222.0.94 () {56 vars in 1093 bytes} [Tue Jul 26 14:42:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:42:45.101455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be9475-5399-4b24-97fc-83ccf4f79cf1 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:45.152066 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:45.255830 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be9475-5399-4b24-97fc-83ccf4f79cf1 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:45.317215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fea865-d533-4bf2-b463-e022e7bc6378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:45.317997 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fea865-d533-4bf2-b463-e022e7bc6378 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:45.318477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fea865-d533-4bf2-b463-e022e7bc6378 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:45.318813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fea865-d533-4bf2-b463-e022e7bc6378 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:45.319839 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fea865-d533-4bf2-b463-e022e7bc6378 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:45.320300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fea865-d533-4bf2-b463-e022e7bc6378 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:45.322245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fea865-d533-4bf2-b463-e022e7bc6378 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:45.345412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 545/1100] 10.222.0.94 () {56 vars in 1133 bytes} [Tue Jul 26 14:42:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:45.352698 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b590710-17cf-4959-ac3c-8158dfef1c9c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:45.395119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:45.506759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b590710-17cf-4959-ac3c-8158dfef1c9c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:45.554841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5492bba-3b55-4c26-967c-d18012c37e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:45.555726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5492bba-3b55-4c26-967c-d18012c37e88 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:45.556020 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5492bba-3b55-4c26-967c-d18012c37e88 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:45.556342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5492bba-3b55-4c26-967c-d18012c37e88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:45.640925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5492bba-3b55-4c26-967c-d18012c37e88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:45.643243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5492bba-3b55-4c26-967c-d18012c37e88 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:45.686662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:45.798909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 556/1101] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:42:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:42:45.805099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fea865-d533-4bf2-b463-e022e7bc6378 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:45.924811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:45.940971 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fea865-d533-4bf2-b463-e022e7bc6378 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:46.000093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be17a0b-5aa7-4fe7-8a84-80acb81945ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:46.001067 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be17a0b-5aa7-4fe7-8a84-80acb81945ed None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:46.001357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be17a0b-5aa7-4fe7-8a84-80acb81945ed None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:46.002062 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be17a0b-5aa7-4fe7-8a84-80acb81945ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:46.025710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be17a0b-5aa7-4fe7-8a84-80acb81945ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:46.027315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be17a0b-5aa7-4fe7-8a84-80acb81945ed None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-1be17a0b-5aa7-4fe7-8a84-80acb81945ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:42:46.047716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:46.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:46.051344 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:42:46.051344 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:42:46.051344 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:42:46.051344 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:42:46.051344 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 546/1102] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:46.056492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5492bba-3b55-4c26-967c-d18012c37e88 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:46.077380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15238bed-ae74-4b3a-bb40-4c4159a2093e None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:46.077766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15238bed-ae74-4b3a-bb40-4c4159a2093e None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:46.078144 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15238bed-ae74-4b3a-bb40-4c4159a2093e None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:46.079625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 547/1103] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:42:46.085739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27dd1ca-4bd6-4c23-88cf-cd5673908204 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:46.086243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27dd1ca-4bd6-4c23-88cf-cd5673908204 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:46.086539 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27dd1ca-4bd6-4c23-88cf-cd5673908204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:46.107111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-d27dd1ca-4bd6-4c23-88cf-cd5673908204 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:46.109167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d27dd1ca-4bd6-4c23-88cf-cd5673908204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:46.198223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 548/1104] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:42:46.215588 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5492bba-3b55-4c26-967c-d18012c37e88 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:46.227050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27dd1ca-4bd6-4c23-88cf-cd5673908204 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:46.256639 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:46.277194 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa104374-93f3-450a-8ad4-3f8686719530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:46.278386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104374-93f3-450a-8ad4-3f8686719530 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:46.280197 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104374-93f3-450a-8ad4-3f8686719530 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:46.304869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa104374-93f3-450a-8ad4-3f8686719530 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:46.304869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa104374-93f3-450a-8ad4-3f8686719530 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:46.304869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa104374-93f3-450a-8ad4-3f8686719530 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:46.304869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa104374-93f3-450a-8ad4-3f8686719530 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:46.308068 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 557/1105] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:46.314438 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:46.316271 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa104374-93f3-450a-8ad4-3f8686719530 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:46.352270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27dd1ca-4bd6-4c23-88cf-cd5673908204 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:46.395076 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95139ee0-740a-41c4-9a80-942aaf926ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:46.395785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95139ee0-740a-41c4-9a80-942aaf926ed2 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:46.396067 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95139ee0-740a-41c4-9a80-942aaf926ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:46.396346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95139ee0-740a-41c4-9a80-942aaf926ed2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:46.397618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95139ee0-740a-41c4-9a80-942aaf926ed2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:46.399071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95139ee0-740a-41c4-9a80-942aaf926ed2 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:46.401129 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 549/1106] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:46.410507 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95139ee0-740a-41c4-9a80-942aaf926ed2 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:46.453345 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa104374-93f3-450a-8ad4-3f8686719530 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:46.498636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1c4a7-af8d-43ac-baa0-87231f41fa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:46.499286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1c4a7-af8d-43ac-baa0-87231f41fa93 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:46.499588 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1c4a7-af8d-43ac-baa0-87231f41fa93 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:46.499858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1c4a7-af8d-43ac-baa0-87231f41fa93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:46.508934 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1c4a7-af8d-43ac-baa0-87231f41fa93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:46.510866 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1c4a7-af8d-43ac-baa0-87231f41fa93 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-acd1c4a7-af8d-43ac-baa0-87231f41fa93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:42:46.523591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:42:46.525416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:42:46.527430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:42:46.527430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:42:46.527430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:42:46.527430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:42:46.527430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:42:46.527430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:42:46.527430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:42:46.527430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:42:46.527430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 558/1107] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:42:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:42:46.530912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1c4a7-af8d-43ac-baa0-87231f41fa93 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:46.546516 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95139ee0-740a-41c4-9a80-942aaf926ed2 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:46.589181 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab5541-3d04-40a9-bc11-7e0347ce70b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:46.590095 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab5541-3d04-40a9-bc11-7e0347ce70b8 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:46.590620 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab5541-3d04-40a9-bc11-7e0347ce70b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:46.591078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab5541-3d04-40a9-bc11-7e0347ce70b8 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:46.593099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab5541-3d04-40a9-bc11-7e0347ce70b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:46.595013 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab5541-3d04-40a9-bc11-7e0347ce70b8 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:46.596596 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:46.612027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:46.664675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1c4a7-af8d-43ac-baa0-87231f41fa93 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:46.707277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75758217-d2dd-4b35-9fa5-7ed06d49729b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:46.707900 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75758217-d2dd-4b35-9fa5-7ed06d49729b None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:46.708178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75758217-d2dd-4b35-9fa5-7ed06d49729b None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:46.708441 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75758217-d2dd-4b35-9fa5-7ed06d49729b None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:46.709206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75758217-d2dd-4b35-9fa5-7ed06d49729b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:46.709511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75758217-d2dd-4b35-9fa5-7ed06d49729b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:46.710962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75758217-d2dd-4b35-9fa5-7ed06d49729b None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:46.725591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 559/1108] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:42:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:46.732699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75758217-d2dd-4b35-9fa5-7ed06d49729b None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:46.820386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 550/1109] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:42:46 2022] POST /identity/v3/users => generated 286 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:42:46.874819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75758217-d2dd-4b35-9fa5-7ed06d49729b None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:46.877048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:46.922965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c7e0e-e2de-4b13-aeb2-850a2434adf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:46.923648 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c7e0e-e2de-4b13-aeb2-850a2434adf1 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:46.923936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c7e0e-e2de-4b13-aeb2-850a2434adf1 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:46.924207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c7e0e-e2de-4b13-aeb2-850a2434adf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:46.925041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c7e0e-e2de-4b13-aeb2-850a2434adf1 None admin] RBAC: Adding query filter params (role.id=57e2f79cce1a4685848082d34f1eb041, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=6a4278fe995e432d88f4346d6f54218b) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:46.925351 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c7e0e-e2de-4b13-aeb2-850a2434adf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:46.926851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c7e0e-e2de-4b13-aeb2-850a2434adf1 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:46.933848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 560/1110] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:42:46 2022] GET /identity/v3/role_assignments?user.id=6a4278fe995e432d88f4346d6f54218b&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=57e2f79cce1a4685848082d34f1eb041 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:49.048554 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7a9c4-6cfd-4148-a09f-d698a89caa95 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:49.049196 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7a9c4-6cfd-4148-a09f-d698a89caa95 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:49.049526 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7a9c4-6cfd-4148-a09f-d698a89caa95 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:49.051059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 551/1111] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:49.057192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94546a-6afd-4b40-9f8f-d4cdb39a7ef7 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:49.057736 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94546a-6afd-4b40-9f8f-d4cdb39a7ef7 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:49.057736 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94546a-6afd-4b40-9f8f-d4cdb39a7ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:49.078643 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-3d94546a-6afd-4b40-9f8f-d4cdb39a7ef7 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:49.079896 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3d94546a-6afd-4b40-9f8f-d4cdb39a7ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:49.160705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 561/1112] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:49.185785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7a9c4-6cfd-4148-a09f-d698a89caa95 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:49.324552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7a9c4-6cfd-4148-a09f-d698a89caa95 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:49.367928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b56dfe-3770-400f-8a2d-4518adfed84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:49.368594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b56dfe-3770-400f-8a2d-4518adfed84c None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:49.368878 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b56dfe-3770-400f-8a2d-4518adfed84c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:49.369154 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b56dfe-3770-400f-8a2d-4518adfed84c None admin] PATH_INFO: `/v3/roles/eff5cca513be4ed5a9b001c0559fb476` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:49.370755 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b56dfe-3770-400f-8a2d-4518adfed84c None admin] RBAC: Authorizing `identity:get_role(role_id=eff5cca513be4ed5a9b001c0559fb476)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:49.372194 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b56dfe-3770-400f-8a2d-4518adfed84c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:49.373670 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 552/1113] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:42:49 2022] GET /identity/v3/roles/eff5cca513be4ed5a9b001c0559fb476 => generated 223 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:49.379979 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94546a-6afd-4b40-9f8f-d4cdb39a7ef7 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:49.501310 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:49.508112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94546a-6afd-4b40-9f8f-d4cdb39a7ef7 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:49.550716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6558fa-9be8-4b63-8caf-38a6af83d38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:49.551369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6558fa-9be8-4b63-8caf-38a6af83d38d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:49.551690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6558fa-9be8-4b63-8caf-38a6af83d38d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:49.551984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6558fa-9be8-4b63-8caf-38a6af83d38d None admin] PATH_INFO: `/v3/users/f09b8aa8a2cf4f6e800e8db6598de383` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:49.569517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6558fa-9be8-4b63-8caf-38a6af83d38d None admin] RBAC: Authorizing `identity:get_user(user_id=f09b8aa8a2cf4f6e800e8db6598de383)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:49.571062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6558fa-9be8-4b63-8caf-38a6af83d38d None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:49.572925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 562/1114] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:42:49 2022] GET /identity/v3/users/f09b8aa8a2cf4f6e800e8db6598de383 => generated 286 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:49.580229 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b56dfe-3770-400f-8a2d-4518adfed84c None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:49.658813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:49.708933 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b56dfe-3770-400f-8a2d-4518adfed84c None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:49.754397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbae60-11c6-425d-ad03-4000a17dbd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:49.755016 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbae60-11c6-425d-ad03-4000a17dbd6a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:49.755363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbae60-11c6-425d-ad03-4000a17dbd6a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:49.755727 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbae60-11c6-425d-ad03-4000a17dbd6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:49.859715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbae60-11c6-425d-ad03-4000a17dbd6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:49.861207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbae60-11c6-425d-ad03-4000a17dbd6a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:50.025426 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:50.072305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 553/1115] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:42:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:42:50.080007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6558fa-9be8-4b63-8caf-38a6af83d38d None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:50.238482 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6558fa-9be8-4b63-8caf-38a6af83d38d None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:50.289888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8c6af-983d-4449-8d84-4b81f46a3df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:50.290587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8c6af-983d-4449-8d84-4b81f46a3df0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:50.290859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8c6af-983d-4449-8d84-4b81f46a3df0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:50.291129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8c6af-983d-4449-8d84-4b81f46a3df0 None admin] PATH_INFO: `/v3/projects/be587c83303a4dcc9b6deb622caf11da` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:50.292460 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8c6af-983d-4449-8d84-4b81f46a3df0 None admin] RBAC: Authorizing `identity:get_project(project_id=be587c83303a4dcc9b6deb622caf11da)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:50.293933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8c6af-983d-4449-8d84-4b81f46a3df0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:50.297152 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 563/1116] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:42:50 2022] GET /identity/v3/projects/be587c83303a4dcc9b6deb622caf11da => generated 311 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:50.304556 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbae60-11c6-425d-ad03-4000a17dbd6a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:50.431260 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbae60-11c6-425d-ad03-4000a17dbd6a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:50.472176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2052694-49b2-4bc8-ad69-25e5c14dbbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:50.472797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2052694-49b2-4bc8-ad69-25e5c14dbbb2 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:50.473080 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2052694-49b2-4bc8-ad69-25e5c14dbbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:50.473357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2052694-49b2-4bc8-ad69-25e5c14dbbb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:50.474289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2052694-49b2-4bc8-ad69-25e5c14dbbb2 None admin] RBAC: Adding query filter params (role.id=eff5cca513be4ed5a9b001c0559fb476, scope.project.id=be587c83303a4dcc9b6deb622caf11da, user.id=f09b8aa8a2cf4f6e800e8db6598de383) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:50.474720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2052694-49b2-4bc8-ad69-25e5c14dbbb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:50.476120 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2052694-49b2-4bc8-ad69-25e5c14dbbb2 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:50.483367 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 554/1117] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:42:50 2022] GET /identity/v3/role_assignments?user.id=f09b8aa8a2cf4f6e800e8db6598de383&scope.project.id=be587c83303a4dcc9b6deb622caf11da&role.id=eff5cca513be4ed5a9b001c0559fb476 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:52.734769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146446c-1eb6-4466-8928-4c3c6db5b743 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:52.735559 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146446c-1eb6-4466-8928-4c3c6db5b743 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:52.735726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146446c-1eb6-4466-8928-4c3c6db5b743 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:52.737411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 564/1118] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:52.745464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef688831-a2c8-4921-9a1d-4525ea610fde None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:52.746297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef688831-a2c8-4921-9a1d-4525ea610fde None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:52.746447 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef688831-a2c8-4921-9a1d-4525ea610fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:52.769377 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ef688831-a2c8-4921-9a1d-4525ea610fde None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:52.770669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ef688831-a2c8-4921-9a1d-4525ea610fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:52.798841 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:52.876707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 555/1119] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:52.916192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146446c-1eb6-4466-8928-4c3c6db5b743 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:52.945517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:53.036090 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146446c-1eb6-4466-8928-4c3c6db5b743 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:53.076195 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a5c86-187a-4d00-9967-715c784a95c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:53.077048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a5c86-187a-4d00-9967-715c784a95c6 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:53.077504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a5c86-187a-4d00-9967-715c784a95c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:53.077791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a5c86-187a-4d00-9967-715c784a95c6 None admin] PATH_INFO: `/v3/roles/eff5cca513be4ed5a9b001c0559fb476` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:53.080034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a5c86-187a-4d00-9967-715c784a95c6 None admin] RBAC: Authorizing `identity:get_role(role_id=eff5cca513be4ed5a9b001c0559fb476)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:53.082819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a5c86-187a-4d00-9967-715c784a95c6 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:53.085482 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 565/1120] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:42:52 2022] GET /identity/v3/roles/eff5cca513be4ed5a9b001c0559fb476 => generated 223 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:53.093603 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef688831-a2c8-4921-9a1d-4525ea610fde None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:53.211348 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef688831-a2c8-4921-9a1d-4525ea610fde None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:53.254574 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb400ee-c73c-481f-867e-7b76c5fd0075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:53.255253 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb400ee-c73c-481f-867e-7b76c5fd0075 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:53.255573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb400ee-c73c-481f-867e-7b76c5fd0075 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:53.255853 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb400ee-c73c-481f-867e-7b76c5fd0075 None admin] PATH_INFO: `/v3/users/f09b8aa8a2cf4f6e800e8db6598de383` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:53.257196 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb400ee-c73c-481f-867e-7b76c5fd0075 None admin] RBAC: Authorizing `identity:get_user(user_id=f09b8aa8a2cf4f6e800e8db6598de383)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:53.258671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb400ee-c73c-481f-867e-7b76c5fd0075 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:53.260478 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 556/1121] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:42:53 2022] GET /identity/v3/users/f09b8aa8a2cf4f6e800e8db6598de383 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:53.270690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a5c86-187a-4d00-9967-715c784a95c6 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:53.384284 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:53.396307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a5c86-187a-4d00-9967-715c784a95c6 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:53.439318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee90e6f-2a5c-4c30-aad7-22c868d1ae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:53.439992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee90e6f-2a5c-4c30-aad7-22c868d1ae6a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:53.440275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee90e6f-2a5c-4c30-aad7-22c868d1ae6a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:53.440549 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee90e6f-2a5c-4c30-aad7-22c868d1ae6a None admin] PATH_INFO: `/v3/projects/be587c83303a4dcc9b6deb622caf11da` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:53.441802 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee90e6f-2a5c-4c30-aad7-22c868d1ae6a None admin] RBAC: Authorizing `identity:get_project(project_id=be587c83303a4dcc9b6deb622caf11da)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:53.443307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee90e6f-2a5c-4c30-aad7-22c868d1ae6a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:53.445184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 566/1122] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:42:53 2022] GET /identity/v3/projects/be587c83303a4dcc9b6deb622caf11da => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:53.452662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb400ee-c73c-481f-867e-7b76c5fd0075 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:53.500523 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:53.622405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb400ee-c73c-481f-867e-7b76c5fd0075 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:53.663857 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1809c2c-7631-4f27-ba7e-f17cf29ffe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:53.664460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1809c2c-7631-4f27-ba7e-f17cf29ffe3a None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:53.664744 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1809c2c-7631-4f27-ba7e-f17cf29ffe3a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:53.665024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1809c2c-7631-4f27-ba7e-f17cf29ffe3a None admin] PATH_INFO: `/v3/projects/be587c83303a4dcc9b6deb622caf11da/users/f09b8aa8a2cf4f6e800e8db6598de383/roles/eff5cca513be4ed5a9b001c0559fb476` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:53.667514 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1809c2c-7631-4f27-ba7e-f17cf29ffe3a None admin] RBAC: Authorizing `identity:create_grant(project_id=be587c83303a4dcc9b6deb622caf11da, user_id=f09b8aa8a2cf4f6e800e8db6598de383, role_id=eff5cca513be4ed5a9b001c0559fb476)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:53.669033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1809c2c-7631-4f27-ba7e-f17cf29ffe3a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:53.789168 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 557/1123] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:42:53 2022] PUT /identity/v3/projects/be587c83303a4dcc9b6deb622caf11da/users/f09b8aa8a2cf4f6e800e8db6598de383/roles/eff5cca513be4ed5a9b001c0559fb476 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:42:55.937796 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7907c0d-d4f7-46f7-ad9c-a7e9e82c2243 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:55.938338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7907c0d-d4f7-46f7-ad9c-a7e9e82c2243 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:55.938338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7907c0d-d4f7-46f7-ad9c-a7e9e82c2243 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:55.940370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 567/1124] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:55.945463 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0761f-bfa7-41a9-ba26-09b62b865584 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:55.946024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0761f-bfa7-41a9-ba26-09b62b865584 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:55.946145 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0761f-bfa7-41a9-ba26-09b62b865584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:55.965652 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-37d0761f-bfa7-41a9-ba26-09b62b865584 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:55.968015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-37d0761f-bfa7-41a9-ba26-09b62b865584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:56.059186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 558/1125] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:56.085845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7907c0d-d4f7-46f7-ad9c-a7e9e82c2243 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:56.239648 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7907c0d-d4f7-46f7-ad9c-a7e9e82c2243 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:56.293732 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07b86c-d165-4f98-8927-62fd93d948c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:56.294948 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07b86c-d165-4f98-8927-62fd93d948c3 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:56.295227 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07b86c-d165-4f98-8927-62fd93d948c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:56.295537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07b86c-d165-4f98-8927-62fd93d948c3 None admin] PATH_INFO: `/v3/roles/eff5cca513be4ed5a9b001c0559fb476` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:56.297181 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07b86c-d165-4f98-8927-62fd93d948c3 None admin] RBAC: Authorizing `identity:get_role(role_id=eff5cca513be4ed5a9b001c0559fb476)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:56.298578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07b86c-d165-4f98-8927-62fd93d948c3 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:56.300026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 568/1126] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:42:56 2022] GET /identity/v3/roles/eff5cca513be4ed5a9b001c0559fb476 => generated 223 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:56.306477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0761f-bfa7-41a9-ba26-09b62b865584 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:56.324136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:56.433345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0761f-bfa7-41a9-ba26-09b62b865584 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:56.480547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b5c97-bccc-4bf5-bcea-f2d6b178f724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:56.482149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b5c97-bccc-4bf5-bcea-f2d6b178f724 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:56.482653 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b5c97-bccc-4bf5-bcea-f2d6b178f724 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:56.483159 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b5c97-bccc-4bf5-bcea-f2d6b178f724 None admin] PATH_INFO: `/v3/users/f09b8aa8a2cf4f6e800e8db6598de383` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:56.485227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b5c97-bccc-4bf5-bcea-f2d6b178f724 None admin] RBAC: Authorizing `identity:get_user(user_id=f09b8aa8a2cf4f6e800e8db6598de383)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:56.486839 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b5c97-bccc-4bf5-bcea-f2d6b178f724 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:56.488827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 559/1127] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:42:56 2022] GET /identity/v3/users/f09b8aa8a2cf4f6e800e8db6598de383 => generated 286 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:42:56.495290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07b86c-d165-4f98-8927-62fd93d948c3 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:56.529303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:56.622935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07b86c-d165-4f98-8927-62fd93d948c3 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:56.665499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc7536-0754-485d-a9eb-c2bc7513c810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:56.666133 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc7536-0754-485d-a9eb-c2bc7513c810 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:56.666406 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc7536-0754-485d-a9eb-c2bc7513c810 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:56.666675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc7536-0754-485d-a9eb-c2bc7513c810 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:56.782611 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc7536-0754-485d-a9eb-c2bc7513c810 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:56.784479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc7536-0754-485d-a9eb-c2bc7513c810 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:56.816314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:56.960414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 569/1128] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:42:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:42:56.967382 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b5c97-bccc-4bf5-bcea-f2d6b178f724 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:57.071379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:57.106941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b5c97-bccc-4bf5-bcea-f2d6b178f724 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:57.172445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65447227-bd36-4233-838d-6faa97d6fece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:57.173484 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65447227-bd36-4233-838d-6faa97d6fece None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:57.173962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65447227-bd36-4233-838d-6faa97d6fece None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:57.174443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65447227-bd36-4233-838d-6faa97d6fece None admin] PATH_INFO: `/v3/projects/be587c83303a4dcc9b6deb622caf11da` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:57.176747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65447227-bd36-4233-838d-6faa97d6fece None admin] RBAC: Authorizing `identity:get_project(project_id=be587c83303a4dcc9b6deb622caf11da)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:57.179523 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65447227-bd36-4233-838d-6faa97d6fece None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:57.182744 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 560/1129] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:42:56 2022] GET /identity/v3/projects/be587c83303a4dcc9b6deb622caf11da => generated 311 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:57.190346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc7536-0754-485d-a9eb-c2bc7513c810 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:57.332895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc7536-0754-485d-a9eb-c2bc7513c810 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:57.364459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:42:57.374110 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592258c-1634-4f9f-8f9e-867f3ae0016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:57.375136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592258c-1634-4f9f-8f9e-867f3ae0016a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:57.375136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592258c-1634-4f9f-8f9e-867f3ae0016a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:57.375563 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592258c-1634-4f9f-8f9e-867f3ae0016a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:57.376428 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592258c-1634-4f9f-8f9e-867f3ae0016a None admin] RBAC: Adding query filter params (role.id=eff5cca513be4ed5a9b001c0559fb476, scope.project.id=be587c83303a4dcc9b6deb622caf11da, user.id=f09b8aa8a2cf4f6e800e8db6598de383) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:42:57.376781 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592258c-1634-4f9f-8f9e-867f3ae0016a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:57.378296 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592258c-1634-4f9f-8f9e-867f3ae0016a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:57.385010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 570/1130] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:42:57 2022] GET /identity/v3/role_assignments?user.id=f09b8aa8a2cf4f6e800e8db6598de383&scope.project.id=be587c83303a4dcc9b6deb622caf11da&role.id=eff5cca513be4ed5a9b001c0559fb476 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:59.587061 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80e3d9-f51e-4fe8-a0d8-1ddf93a9259a None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:59.587821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80e3d9-f51e-4fe8-a0d8-1ddf93a9259a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:59.587821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80e3d9-f51e-4fe8-a0d8-1ddf93a9259a None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:59.589103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 561/1131] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:42:59.596174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bc1cf-4a5e-4097-8343-0956ab252521 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:59.596777 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bc1cf-4a5e-4097-8343-0956ab252521 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:59.596777 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bc1cf-4a5e-4097-8343-0956ab252521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:59.620016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c38bc1cf-4a5e-4097-8343-0956ab252521 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:42:59.621369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c38bc1cf-4a5e-4097-8343-0956ab252521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:42:59.713530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 571/1132] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:42:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:42:59.739149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80e3d9-f51e-4fe8-a0d8-1ddf93a9259a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:42:59.857637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80e3d9-f51e-4fe8-a0d8-1ddf93a9259a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:42:59.898163 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f99f0-4cbd-45f8-bab9-037bdf63fb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:42:59.898818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f99f0-4cbd-45f8-bab9-037bdf63fb28 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:42:59.899098 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f99f0-4cbd-45f8-bab9-037bdf63fb28 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:42:59.899367 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f99f0-4cbd-45f8-bab9-037bdf63fb28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:42:59.900654 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f99f0-4cbd-45f8-bab9-037bdf63fb28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:42:59.902115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f99f0-4cbd-45f8-bab9-037bdf63fb28 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:42:59.904565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 562/1133] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:42:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:42:59.911212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38bc1cf-4a5e-4097-8343-0956ab252521 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:00.037598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38bc1cf-4a5e-4097-8343-0956ab252521 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:00.062099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:00.087771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b48325-dd73-401e-a021-b743b6831b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:00.088411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b48325-dd73-401e-a021-b743b6831b0c None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:00.088703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b48325-dd73-401e-a021-b743b6831b0c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:00.088959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b48325-dd73-401e-a021-b743b6831b0c None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:00.089672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b48325-dd73-401e-a021-b743b6831b0c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:00.091428 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b48325-dd73-401e-a021-b743b6831b0c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:00.212440 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 572/1134] 10.222.0.94 () {60 vars in 1141 bytes} [Tue Jul 26 14:42:59 2022] POST /identity/v3/projects => generated 311 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:43:02.654294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9921c-5264-4b36-83d4-7564eca4f6d4 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:02.654962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9921c-5264-4b36-83d4-7564eca4f6d4 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:02.655241 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9921c-5264-4b36-83d4-7564eca4f6d4 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:02.658288 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 563/1135] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:43:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:02.664281 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342b0a6-4fa8-49b0-82cc-69d90ca598b6 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:02.665124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342b0a6-4fa8-49b0-82cc-69d90ca598b6 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:02.665124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342b0a6-4fa8-49b0-82cc-69d90ca598b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:02.686222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-4342b0a6-4fa8-49b0-82cc-69d90ca598b6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:02.687544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4342b0a6-4fa8-49b0-82cc-69d90ca598b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:02.777025 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 573/1136] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:43:02.805346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9921c-5264-4b36-83d4-7564eca4f6d4 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:02.828761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:02.934267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9921c-5264-4b36-83d4-7564eca4f6d4 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:02.977425 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ea4b7-16d5-40b2-8134-b5de25e25d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:02.978231 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ea4b7-16d5-40b2-8134-b5de25e25d2e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:02.978620 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ea4b7-16d5-40b2-8134-b5de25e25d2e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:02.978998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ea4b7-16d5-40b2-8134-b5de25e25d2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:02.980484 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ea4b7-16d5-40b2-8134-b5de25e25d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:02.982809 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ea4b7-16d5-40b2-8134-b5de25e25d2e None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:02.984639 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 564/1137] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:43:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:02.994323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342b0a6-4fa8-49b0-82cc-69d90ca598b6 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:03.194200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342b0a6-4fa8-49b0-82cc-69d90ca598b6 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:03.197277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:03.254557 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e14c55-b524-4a01-a48c-802f19367b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:03.255257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e14c55-b524-4a01-a48c-802f19367b62 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:03.255697 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e14c55-b524-4a01-a48c-802f19367b62 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:03.256067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e14c55-b524-4a01-a48c-802f19367b62 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:03.256955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e14c55-b524-4a01-a48c-802f19367b62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:03.258540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e14c55-b524-4a01-a48c-802f19367b62 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:03.378169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 574/1138] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:43:02 2022] POST /identity/v3/users => generated 286 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:43:05.638468 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c2739-2466-48cd-b1be-d4d9dfa59bb0 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:05.639089 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c2739-2466-48cd-b1be-d4d9dfa59bb0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:05.639089 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c2739-2466-48cd-b1be-d4d9dfa59bb0 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:05.640586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 565/1139] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:43:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:43:05.648135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb30adf-a3ce-45a4-b289-96c2fbdcdcc5 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:05.648675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb30adf-a3ce-45a4-b289-96c2fbdcdcc5 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:05.649137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb30adf-a3ce-45a4-b289-96c2fbdcdcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:05.684233 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-5bb30adf-a3ce-45a4-b289-96c2fbdcdcc5 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:05.686554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5bb30adf-a3ce-45a4-b289-96c2fbdcdcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:05.816845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 575/1140] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:43:05.844077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c2739-2466-48cd-b1be-d4d9dfa59bb0 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:05.951935 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:05.980774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c2739-2466-48cd-b1be-d4d9dfa59bb0 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:06.032744 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3076b-9903-4ea4-8188-456deb0f5884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:06.033405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3076b-9903-4ea4-8188-456deb0f5884 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:06.033687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3076b-9903-4ea4-8188-456deb0f5884 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:06.033961 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3076b-9903-4ea4-8188-456deb0f5884 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:06.047353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3076b-9903-4ea4-8188-456deb0f5884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-41d3076b-9903-4ea4-8188-456deb0f5884 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:43:06.052868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:43:06.054429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:43:06.055806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:43:06.055806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:43:06.055806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:43:06.055806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:43:06.055806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:43:06.055806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:43:06.055806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:43:06.055806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:43:06.055806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:43:06.055806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:43:06.055806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:43:06.055806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:43:06.055806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 566/1141] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:43:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:43:06.065865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb30adf-a3ce-45a4-b289-96c2fbdcdcc5 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:06.222517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:06.236230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb30adf-a3ce-45a4-b289-96c2fbdcdcc5 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:06.281593 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba25a8-ce25-471c-9fbd-c397cd1425c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:06.282226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba25a8-ce25-471c-9fbd-c397cd1425c5 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:06.282761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba25a8-ce25-471c-9fbd-c397cd1425c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:06.283037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba25a8-ce25-471c-9fbd-c397cd1425c5 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:06.283800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba25a8-ce25-471c-9fbd-c397cd1425c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:43:06.284108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba25a8-ce25-471c-9fbd-c397cd1425c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:06.285617 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba25a8-ce25-471c-9fbd-c397cd1425c5 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:06.296117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 576/1142] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:43:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:06.302505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3076b-9903-4ea4-8188-456deb0f5884 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:06.428459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3076b-9903-4ea4-8188-456deb0f5884 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:06.472837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c4f23-77ac-4178-81b1-e4fb4440c04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:06.473409 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c4f23-77ac-4178-81b1-e4fb4440c04b None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:06.473693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c4f23-77ac-4178-81b1-e4fb4440c04b None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:06.473974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c4f23-77ac-4178-81b1-e4fb4440c04b None admin] PATH_INFO: `/v3/users/fc2ccefc302046179402635c6573c590` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:06.491434 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c4f23-77ac-4178-81b1-e4fb4440c04b None admin] RBAC: Authorizing `identity:get_user(user_id=fc2ccefc302046179402635c6573c590)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:06.493240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c4f23-77ac-4178-81b1-e4fb4440c04b None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:06.495144 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 567/1143] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:43:06 2022] GET /identity/v3/users/fc2ccefc302046179402635c6573c590 => generated 286 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:43:06.501676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba25a8-ce25-471c-9fbd-c397cd1425c5 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:06.636016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba25a8-ce25-471c-9fbd-c397cd1425c5 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:06.680791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7fe91-d80b-4ce9-879c-61db481b3e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:06.681544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7fe91-d80b-4ce9-879c-61db481b3e10 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:06.681674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7fe91-d80b-4ce9-879c-61db481b3e10 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:06.681972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7fe91-d80b-4ce9-879c-61db481b3e10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:06.707607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:06.767812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7fe91-d80b-4ce9-879c-61db481b3e10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:06.769308 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7fe91-d80b-4ce9-879c-61db481b3e10 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:06.894993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 577/1144] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:43:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:43:06.901362 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c4f23-77ac-4178-81b1-e4fb4440c04b None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:06.906723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:07.027668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c4f23-77ac-4178-81b1-e4fb4440c04b None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:07.094491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efae12-6a68-4c1b-8ad1-6af772246bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:07.095640 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efae12-6a68-4c1b-8ad1-6af772246bab None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:07.096152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efae12-6a68-4c1b-8ad1-6af772246bab None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:07.096652 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efae12-6a68-4c1b-8ad1-6af772246bab None admin] PATH_INFO: `/v3/projects/95ce741e429a460eb91a587dce9053e8` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:07.098960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efae12-6a68-4c1b-8ad1-6af772246bab None admin] RBAC: Authorizing `identity:get_project(project_id=95ce741e429a460eb91a587dce9053e8)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:07.101670 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efae12-6a68-4c1b-8ad1-6af772246bab None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:07.105094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 568/1145] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:43:06 2022] GET /identity/v3/projects/95ce741e429a460eb91a587dce9053e8 => generated 311 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:43:07.112625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7fe91-d80b-4ce9-879c-61db481b3e10 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:07.174576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:07.238501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7fe91-d80b-4ce9-879c-61db481b3e10 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:07.285740 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab5965-c0dd-4c81-9e0f-d873cefd77a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:07.286359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab5965-c0dd-4c81-9e0f-d873cefd77a0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:07.286740 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab5965-c0dd-4c81-9e0f-d873cefd77a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:07.287060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab5965-c0dd-4c81-9e0f-d873cefd77a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:07.288059 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab5965-c0dd-4c81-9e0f-d873cefd77a0 None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=95ce741e429a460eb91a587dce9053e8, user.id=fc2ccefc302046179402635c6573c590) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:43:07.288416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab5965-c0dd-4c81-9e0f-d873cefd77a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:07.290066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab5965-c0dd-4c81-9e0f-d873cefd77a0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:07.297696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 578/1146] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:43:07 2022] GET /identity/v3/role_assignments?user.id=fc2ccefc302046179402635c6573c590&scope.project.id=95ce741e429a460eb91a587dce9053e8&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:09.469631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b1138-8394-418b-a6e1-3d852f20942a None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:09.470235 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b1138-8394-418b-a6e1-3d852f20942a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:09.470235 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b1138-8394-418b-a6e1-3d852f20942a None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:09.471931 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 569/1147] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:09.479140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924593de-69e2-488f-a4ef-12a4ca45ae4d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:09.479999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924593de-69e2-488f-a4ef-12a4ca45ae4d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:09.479999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924593de-69e2-488f-a4ef-12a4ca45ae4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:09.502996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-924593de-69e2-488f-a4ef-12a4ca45ae4d None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:09.504161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-924593de-69e2-488f-a4ef-12a4ca45ae4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:09.597531 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 579/1148] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:43:09.624738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b1138-8394-418b-a6e1-3d852f20942a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:09.719792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:09.757571 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b1138-8394-418b-a6e1-3d852f20942a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:09.802435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e1dde-d58f-4395-886f-fa97cebde9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:09.803298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e1dde-d58f-4395-886f-fa97cebde9eb None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:09.803673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e1dde-d58f-4395-886f-fa97cebde9eb None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:09.804003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e1dde-d58f-4395-886f-fa97cebde9eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:09.817535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e1dde-d58f-4395-886f-fa97cebde9eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-0b0e1dde-d58f-4395-886f-fa97cebde9eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:43:09.822929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:43:09.825538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:43:09.827416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:43:09.827416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:43:09.827416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:43:09.827416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:43:09.827416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:43:09.827416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:43:09.827416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:43:09.827416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:43:09.827416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:43:09.827416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:43:09.827416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:43:09.827416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:43:09.827416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 570/1149] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:43:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:43:09.832131 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924593de-69e2-488f-a4ef-12a4ca45ae4d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:09.923066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:10.032295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924593de-69e2-488f-a4ef-12a4ca45ae4d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:10.098549 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38188152-d7c7-46db-ae23-dc0d5964c9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:10.099657 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38188152-d7c7-46db-ae23-dc0d5964c9c5 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:10.100159 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38188152-d7c7-46db-ae23-dc0d5964c9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:10.100658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38188152-d7c7-46db-ae23-dc0d5964c9c5 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:10.101975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38188152-d7c7-46db-ae23-dc0d5964c9c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:43:10.102616 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38188152-d7c7-46db-ae23-dc0d5964c9c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:10.105088 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38188152-d7c7-46db-ae23-dc0d5964c9c5 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:10.122971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 580/1150] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:43:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:10.129530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e1dde-d58f-4395-886f-fa97cebde9eb None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:10.256978 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e1dde-d58f-4395-886f-fa97cebde9eb None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:10.300533 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:10.302287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76b7e0-6d04-4e4e-bc77-97eb86f20d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:10.302963 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76b7e0-6d04-4e4e-bc77-97eb86f20d18 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:10.303326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76b7e0-6d04-4e4e-bc77-97eb86f20d18 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:10.303815 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76b7e0-6d04-4e4e-bc77-97eb86f20d18 None admin] PATH_INFO: `/v3/users/fc2ccefc302046179402635c6573c590` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:10.305253 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76b7e0-6d04-4e4e-bc77-97eb86f20d18 None admin] RBAC: Authorizing `identity:get_user(user_id=fc2ccefc302046179402635c6573c590)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:10.306766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76b7e0-6d04-4e4e-bc77-97eb86f20d18 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:10.308553 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 571/1151] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:43:10 2022] GET /identity/v3/users/fc2ccefc302046179402635c6573c590 => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:43:10.315668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38188152-d7c7-46db-ae23-dc0d5964c9c5 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:10.448946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38188152-d7c7-46db-ae23-dc0d5964c9c5 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:10.457586 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:10.493514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55676a72-b771-4851-adcc-9933f14b36f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:10.494703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55676a72-b771-4851-adcc-9933f14b36f7 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:10.494703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55676a72-b771-4851-adcc-9933f14b36f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:10.495064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55676a72-b771-4851-adcc-9933f14b36f7 None admin] PATH_INFO: `/v3/projects/95ce741e429a460eb91a587dce9053e8` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:10.497412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55676a72-b771-4851-adcc-9933f14b36f7 None admin] RBAC: Authorizing `identity:get_project(project_id=95ce741e429a460eb91a587dce9053e8)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:10.499456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55676a72-b771-4851-adcc-9933f14b36f7 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:10.501367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 581/1152] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:43:10 2022] GET /identity/v3/projects/95ce741e429a460eb91a587dce9053e8 => generated 311 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:10.509266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76b7e0-6d04-4e4e-bc77-97eb86f20d18 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:10.647003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76b7e0-6d04-4e4e-bc77-97eb86f20d18 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:10.693063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2841db8-dbfc-4e32-a6e5-8abac68478b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:10.694065 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2841db8-dbfc-4e32-a6e5-8abac68478b5 None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:10.694559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2841db8-dbfc-4e32-a6e5-8abac68478b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:10.694915 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2841db8-dbfc-4e32-a6e5-8abac68478b5 None admin] PATH_INFO: `/v3/projects/95ce741e429a460eb91a587dce9053e8/users/fc2ccefc302046179402635c6573c590/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:10.697674 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2841db8-dbfc-4e32-a6e5-8abac68478b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=95ce741e429a460eb91a587dce9053e8, user_id=fc2ccefc302046179402635c6573c590, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:10.699405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2841db8-dbfc-4e32-a6e5-8abac68478b5 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:10.848352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 572/1153] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:43:10 2022] PUT /identity/v3/projects/95ce741e429a460eb91a587dce9053e8/users/fc2ccefc302046179402635c6573c590/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:43:13.128256 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8cee9-e451-44b1-87e6-1c12a5737852 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:13.129151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8cee9-e451-44b1-87e6-1c12a5737852 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:13.129151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8cee9-e451-44b1-87e6-1c12a5737852 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:13.130776 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 582/1154] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:43:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:43:13.137785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9df798-7904-4fe1-8c44-2f0fd4cea63a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:13.138650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9df798-7904-4fe1-8c44-2f0fd4cea63a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:13.138650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9df798-7904-4fe1-8c44-2f0fd4cea63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:13.159800 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-4e9df798-7904-4fe1-8c44-2f0fd4cea63a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:13.161091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4e9df798-7904-4fe1-8c44-2f0fd4cea63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:13.272780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:13.287236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 573/1155] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:43:13.311852 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8cee9-e451-44b1-87e6-1c12a5737852 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:13.453545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8cee9-e451-44b1-87e6-1c12a5737852 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:13.497201 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf1c14-302c-4aff-b037-cc098a96e636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:13.497863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf1c14-302c-4aff-b037-cc098a96e636 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:13.498147 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf1c14-302c-4aff-b037-cc098a96e636 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:13.498417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf1c14-302c-4aff-b037-cc098a96e636 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:13.511448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf1c14-302c-4aff-b037-cc098a96e636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-d9bf1c14-302c-4aff-b037-cc098a96e636 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:43:13.516941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:43:13.518767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:43:13.520431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:43:13.520431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:43:13.520431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:43:13.520431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:43:13.520431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:43:13.520431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:43:13.520431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:43:13.520431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:43:13.520431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:43:13.520431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:43:13.520431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:43:13.520431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:43:13.520431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 583/1156] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:43:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:43:13.527376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9df798-7904-4fe1-8c44-2f0fd4cea63a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:13.664932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9df798-7904-4fe1-8c44-2f0fd4cea63a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:13.710898 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9c8bb-a482-414b-9ef0-3c0bf9c27839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:13.711568 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9c8bb-a482-414b-9ef0-3c0bf9c27839 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:13.711854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9c8bb-a482-414b-9ef0-3c0bf9c27839 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:13.712141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9c8bb-a482-414b-9ef0-3c0bf9c27839 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:13.712883 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9c8bb-a482-414b-9ef0-3c0bf9c27839 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:43:13.713193 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9c8bb-a482-414b-9ef0-3c0bf9c27839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:13.714608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9c8bb-a482-414b-9ef0-3c0bf9c27839 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:13.725586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 574/1157] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:43:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:13.735736 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf1c14-302c-4aff-b037-cc098a96e636 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:13.801715 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:13.936014 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf1c14-302c-4aff-b037-cc098a96e636 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:14.007803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbb1b8-eb27-4bc5-ba50-1ec50c8842f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:14.008992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbb1b8-eb27-4bc5-ba50-1ec50c8842f9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:14.009532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbb1b8-eb27-4bc5-ba50-1ec50c8842f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:14.010048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbb1b8-eb27-4bc5-ba50-1ec50c8842f9 None admin] PATH_INFO: `/v3/users/fc2ccefc302046179402635c6573c590` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:14.012346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbb1b8-eb27-4bc5-ba50-1ec50c8842f9 None admin] RBAC: Authorizing `identity:get_user(user_id=fc2ccefc302046179402635c6573c590)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:14.015173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbb1b8-eb27-4bc5-ba50-1ec50c8842f9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:14.018341 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 584/1158] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:43:13 2022] GET /identity/v3/users/fc2ccefc302046179402635c6573c590 => generated 286 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:14.025351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9c8bb-a482-414b-9ef0-3c0bf9c27839 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:14.103366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:14.162609 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9c8bb-a482-414b-9ef0-3c0bf9c27839 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:14.208260 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b870560-6231-47b7-95ea-400e0cc94ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:14.208960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b870560-6231-47b7-95ea-400e0cc94ecf None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:14.209356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b870560-6231-47b7-95ea-400e0cc94ecf None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:14.209603 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b870560-6231-47b7-95ea-400e0cc94ecf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:14.307654 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b870560-6231-47b7-95ea-400e0cc94ecf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:14.311110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b870560-6231-47b7-95ea-400e0cc94ecf None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:14.406329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:14.466329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 575/1159] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:43:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:43:14.473101 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbb1b8-eb27-4bc5-ba50-1ec50c8842f9 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:14.601476 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbb1b8-eb27-4bc5-ba50-1ec50c8842f9 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:14.603390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:14.646699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e12be-69d1-42e0-96a5-b4b7f8f610a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:14.647381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e12be-69d1-42e0-96a5-b4b7f8f610a9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:14.647690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e12be-69d1-42e0-96a5-b4b7f8f610a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:14.648259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e12be-69d1-42e0-96a5-b4b7f8f610a9 None admin] PATH_INFO: `/v3/projects/95ce741e429a460eb91a587dce9053e8` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:14.649526 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e12be-69d1-42e0-96a5-b4b7f8f610a9 None admin] RBAC: Authorizing `identity:get_project(project_id=95ce741e429a460eb91a587dce9053e8)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:14.651038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e12be-69d1-42e0-96a5-b4b7f8f610a9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:14.653577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 585/1160] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:43:14 2022] GET /identity/v3/projects/95ce741e429a460eb91a587dce9053e8 => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:14.664558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b870560-6231-47b7-95ea-400e0cc94ecf None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:14.787361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b870560-6231-47b7-95ea-400e0cc94ecf None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:14.827871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93dc4c-6173-469d-8f48-e194a7fdf8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:14.828498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93dc4c-6173-469d-8f48-e194a7fdf8f3 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:14.828780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93dc4c-6173-469d-8f48-e194a7fdf8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:14.829052 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93dc4c-6173-469d-8f48-e194a7fdf8f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:14.829858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93dc4c-6173-469d-8f48-e194a7fdf8f3 None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=95ce741e429a460eb91a587dce9053e8, user.id=fc2ccefc302046179402635c6573c590) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:43:14.830167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93dc4c-6173-469d-8f48-e194a7fdf8f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:14.831666 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93dc4c-6173-469d-8f48-e194a7fdf8f3 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:14.838629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 576/1161] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:43:14 2022] GET /identity/v3/role_assignments?user.id=fc2ccefc302046179402635c6573c590&scope.project.id=95ce741e429a460eb91a587dce9053e8&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:17.078444 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee7eb4-c543-4afd-8f23-d0a5976c7cd4 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:17.079073 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee7eb4-c543-4afd-8f23-d0a5976c7cd4 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:17.079073 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee7eb4-c543-4afd-8f23-d0a5976c7cd4 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:17.080626 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 586/1162] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:43:17.087403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7db06d-5eae-4908-9c74-4187a47f0091 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:17.088080 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7db06d-5eae-4908-9c74-4187a47f0091 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:17.088394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7db06d-5eae-4908-9c74-4187a47f0091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:17.119868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-1b7db06d-5eae-4908-9c74-4187a47f0091 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:17.122540 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1b7db06d-5eae-4908-9c74-4187a47f0091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:17.145256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:17.238608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 577/1163] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:43:17.263910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee7eb4-c543-4afd-8f23-d0a5976c7cd4 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:17.396798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee7eb4-c543-4afd-8f23-d0a5976c7cd4 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:17.439909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5faf5-d8e5-4c40-afb6-369a965c8b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:17.440598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5faf5-d8e5-4c40-afb6-369a965c8b19 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:17.440879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5faf5-d8e5-4c40-afb6-369a965c8b19 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:17.441156 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5faf5-d8e5-4c40-afb6-369a965c8b19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:17.490426 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:17.526211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5faf5-d8e5-4c40-afb6-369a965c8b19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:17.528411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5faf5-d8e5-4c40-afb6-369a965c8b19 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:17.658627 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 587/1164] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:43:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:43:17.665330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7db06d-5eae-4908-9c74-4187a47f0091 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:17.789705 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7db06d-5eae-4908-9c74-4187a47f0091 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:17.832762 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca3dca-9b85-4eeb-bc9c-75aead8bf329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:17.833445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca3dca-9b85-4eeb-bc9c-75aead8bf329 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:17.833729 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca3dca-9b85-4eeb-bc9c-75aead8bf329 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:17.834008 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca3dca-9b85-4eeb-bc9c-75aead8bf329 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:17.843281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca3dca-9b85-4eeb-bc9c-75aead8bf329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:17.844927 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca3dca-9b85-4eeb-bc9c-75aead8bf329 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-75ca3dca-9b85-4eeb-bc9c-75aead8bf329 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:43:17.859420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:43:17.861404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:43:17.863077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 578/1165] 10.222.0.94 () {56 vars in 1117 bytes} [Tue Jul 26 14:43:17 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:43:17.869210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5faf5-d8e5-4c40-afb6-369a965c8b19 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:17.954099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:17.994261 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5faf5-d8e5-4c40-afb6-369a965c8b19 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:18.051285 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d339b9a-e5ec-403b-a216-1be34d71b00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:18.052013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d339b9a-e5ec-403b-a216-1be34d71b00c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:18.052255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d339b9a-e5ec-403b-a216-1be34d71b00c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:18.052531 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d339b9a-e5ec-403b-a216-1be34d71b00c None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:18.053245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d339b9a-e5ec-403b-a216-1be34d71b00c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:43:18.053554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d339b9a-e5ec-403b-a216-1be34d71b00c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:18.055089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d339b9a-e5ec-403b-a216-1be34d71b00c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:18.064535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 588/1166] 10.222.0.94 () {56 vars in 1115 bytes} [Tue Jul 26 14:43:17 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:20.250117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561faf70-8bec-4b24-9749-a1d8daa9e6ea None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:20.250594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561faf70-8bec-4b24-9749-a1d8daa9e6ea None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:20.250885 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561faf70-8bec-4b24-9749-a1d8daa9e6ea None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:20.252294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 579/1167] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:20.258848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a7d80-5949-4ee6-89a6-854ebf813107 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:20.259511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a7d80-5949-4ee6-89a6-854ebf813107 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:20.259511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a7d80-5949-4ee6-89a6-854ebf813107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:20.282860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-237a7d80-5949-4ee6-89a6-854ebf813107 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:20.284342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-237a7d80-5949-4ee6-89a6-854ebf813107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:20.369359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 589/1168] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:43:20.394030 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561faf70-8bec-4b24-9749-a1d8daa9e6ea None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:20.405075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:20.567084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561faf70-8bec-4b24-9749-a1d8daa9e6ea None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:20.634467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec452ed1-ae31-495c-8e65-45f451183a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:20.635560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec452ed1-ae31-495c-8e65-45f451183a70 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:20.636051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec452ed1-ae31-495c-8e65-45f451183a70 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:20.636608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec452ed1-ae31-495c-8e65-45f451183a70 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:20.639338 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec452ed1-ae31-495c-8e65-45f451183a70 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:20.641955 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec452ed1-ae31-495c-8e65-45f451183a70 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:20.735590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 580/1169] 10.222.0.94 () {60 vars in 1138 bytes} [Tue Jul 26 14:43:20 2022] POST /identity/v3/domains => generated 267 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:43:22.940239 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819673dc-cbb9-480d-9ba4-7fad6785e2fb None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:22.940852 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819673dc-cbb9-480d-9ba4-7fad6785e2fb None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:22.940852 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819673dc-cbb9-480d-9ba4-7fad6785e2fb None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:22.942283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 590/1170] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:43:22.952366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f755102-30d0-4c84-b4f4-9e96dc54908d None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:22.952922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f755102-30d0-4c84-b4f4-9e96dc54908d None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:22.953337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f755102-30d0-4c84-b4f4-9e96dc54908d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:22.984616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-5f755102-30d0-4c84-b4f4-9e96dc54908d None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:22.986745 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-5f755102-30d0-4c84-b4f4-9e96dc54908d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:23.102212 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:23.116557 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 581/1171] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:43:23.141849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819673dc-cbb9-480d-9ba4-7fad6785e2fb None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:23.225886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:23.294733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819673dc-cbb9-480d-9ba4-7fad6785e2fb None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:23.359268 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58629a-01e3-4a2d-ac6c-2e053df7f60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:23.360255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58629a-01e3-4a2d-ac6c-2e053df7f60e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:23.360667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58629a-01e3-4a2d-ac6c-2e053df7f60e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:23.361057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58629a-01e3-4a2d-ac6c-2e053df7f60e None admin] PATH_INFO: `/v3/domains/56f8834743304aaa80f2c695cc4e966f` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:23.375646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58629a-01e3-4a2d-ac6c-2e053df7f60e None admin] RBAC: Authorizing `identity:get_domain(domain_id=56f8834743304aaa80f2c695cc4e966f)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:23.377283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58629a-01e3-4a2d-ac6c-2e053df7f60e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:23.379232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 591/1172] 10.222.0.94 () {56 vars in 1183 bytes} [Tue Jul 26 14:43:23 2022] GET /identity/v3/domains/56f8834743304aaa80f2c695cc4e966f => generated 267 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:23.388742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f755102-30d0-4c84-b4f4-9e96dc54908d None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:23.520019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f755102-30d0-4c84-b4f4-9e96dc54908d None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:23.565898 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e347e54-c01c-47c6-9b19-e4abe18abbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:23.567011 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e347e54-c01c-47c6-9b19-e4abe18abbd8 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:23.567571 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e347e54-c01c-47c6-9b19-e4abe18abbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:23.568125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e347e54-c01c-47c6-9b19-e4abe18abbd8 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:23.569323 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e347e54-c01c-47c6-9b19-e4abe18abbd8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:23.572065 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e347e54-c01c-47c6-9b19-e4abe18abbd8 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:23.755316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 582/1173] 10.222.0.94 () {60 vars in 1141 bytes} [Tue Jul 26 14:43:23 2022] POST /identity/v3/projects => generated 361 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:43:26.896305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdefbe-30f2-475a-b0f9-7c4eb19b13a1 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:26.897204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdefbe-30f2-475a-b0f9-7c4eb19b13a1 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:26.897204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdefbe-30f2-475a-b0f9-7c4eb19b13a1 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:26.898766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 592/1174] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:43:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:26.909190 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7114790-6fb0-4745-9e4d-fdb3e494f7a9 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:26.909817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7114790-6fb0-4745-9e4d-fdb3e494f7a9 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:26.909817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7114790-6fb0-4745-9e4d-fdb3e494f7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:26.930044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-d7114790-6fb0-4745-9e4d-fdb3e494f7a9 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:26.931277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d7114790-6fb0-4745-9e4d-fdb3e494f7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:26.985415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:27.023422 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 583/1175] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:43:27.051440 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdefbe-30f2-475a-b0f9-7c4eb19b13a1 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:27.179082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdefbe-30f2-475a-b0f9-7c4eb19b13a1 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:27.209541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:27.222672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b0d87-38a8-4da9-8961-a03ce3a5a40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:27.223432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b0d87-38a8-4da9-8961-a03ce3a5a40f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:27.223738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b0d87-38a8-4da9-8961-a03ce3a5a40f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:27.224129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b0d87-38a8-4da9-8961-a03ce3a5a40f None admin] PATH_INFO: `/v3/domains/56f8834743304aaa80f2c695cc4e966f` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:27.225427 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b0d87-38a8-4da9-8961-a03ce3a5a40f None admin] RBAC: Authorizing `identity:get_domain(domain_id=56f8834743304aaa80f2c695cc4e966f)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:27.226896 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b0d87-38a8-4da9-8961-a03ce3a5a40f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:27.228680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 593/1176] 10.222.0.94 () {56 vars in 1183 bytes} [Tue Jul 26 14:43:27 2022] GET /identity/v3/domains/56f8834743304aaa80f2c695cc4e966f => generated 267 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:27.236251 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7114790-6fb0-4745-9e4d-fdb3e494f7a9 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:27.365164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7114790-6fb0-4745-9e4d-fdb3e494f7a9 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:27.408505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ecd0a7-1a55-4051-9678-f7e5c0610ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:27.409132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecd0a7-1a55-4051-9678-f7e5c0610ec4 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:27.409414 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecd0a7-1a55-4051-9678-f7e5c0610ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:27.409687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ecd0a7-1a55-4051-9678-f7e5c0610ec4 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:27.410512 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ecd0a7-1a55-4051-9678-f7e5c0610ec4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:27.412118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ecd0a7-1a55-4051-9678-f7e5c0610ec4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:27.517426 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 584/1177] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:43:27 2022] POST /identity/v3/users => generated 311 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 14:43:29.741308 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0620c-713b-48b5-bce0-25efb5786835 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:29.741963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0620c-713b-48b5-bce0-25efb5786835 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:29.741963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0620c-713b-48b5-bce0-25efb5786835 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:29.743620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 594/1178] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:43:29.749924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d17e1-0731-44d9-a119-1e9f273783b8 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:29.750600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d17e1-0731-44d9-a119-1e9f273783b8 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:29.750600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d17e1-0731-44d9-a119-1e9f273783b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:29.779205 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-a00d17e1-0731-44d9-a119-1e9f273783b8 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:29.781173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a00d17e1-0731-44d9-a119-1e9f273783b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:29.830295 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:29.880911 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 585/1179] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:43:29.907066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0620c-713b-48b5-bce0-25efb5786835 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:30.043063 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0620c-713b-48b5-bce0-25efb5786835 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:30.106220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2baf6-7b49-4275-bfac-c47de385d71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:30.107359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2baf6-7b49-4275-bfac-c47de385d71f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:30.107904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2baf6-7b49-4275-bfac-c47de385d71f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:30.108408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2baf6-7b49-4275-bfac-c47de385d71f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:30.128890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2baf6-7b49-4275-bfac-c47de385d71f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-7df2baf6-7b49-4275-bfac-c47de385d71f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:43:30.139359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:43:30.141312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:43:30.143158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:43:30.143158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:43:30.143158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:43:30.143158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:43:30.143158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:43:30.143158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:43:30.143158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:43:30.143158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:43:30.143158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:43:30.143158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:43:30.143158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:43:30.143158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:43:30.143158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 595/1180] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:43:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:43:30.150045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d17e1-0731-44d9-a119-1e9f273783b8 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:30.287810 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d17e1-0731-44d9-a119-1e9f273783b8 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:30.335800 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c8928-2f76-428e-9451-2804ade2d32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:30.336364 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c8928-2f76-428e-9451-2804ade2d32a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:30.336654 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c8928-2f76-428e-9451-2804ade2d32a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:30.336928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c8928-2f76-428e-9451-2804ade2d32a None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:30.337671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c8928-2f76-428e-9451-2804ade2d32a None admin] RBAC: Adding query filter params (name=admin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:43:30.337978 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c8928-2f76-428e-9451-2804ade2d32a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:30.340056 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c8928-2f76-428e-9451-2804ade2d32a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:30.350197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 586/1181] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:43:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:30.360540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2baf6-7b49-4275-bfac-c47de385d71f None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:30.427642 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:30.486412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2baf6-7b49-4275-bfac-c47de385d71f None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:30.528826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c53336-1b13-431e-9d27-2bee6726df4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:30.529479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c53336-1b13-431e-9d27-2bee6726df4c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:30.529754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c53336-1b13-431e-9d27-2bee6726df4c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:30.530127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c53336-1b13-431e-9d27-2bee6726df4c None admin] PATH_INFO: `/v3/users/a6a1a89a33d9413bb6773882a965e98d` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:30.548318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c53336-1b13-431e-9d27-2bee6726df4c None admin] RBAC: Authorizing `identity:get_user(user_id=a6a1a89a33d9413bb6773882a965e98d)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:30.550023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c53336-1b13-431e-9d27-2bee6726df4c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:30.552102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 596/1182] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:43:30 2022] GET /identity/v3/users/a6a1a89a33d9413bb6773882a965e98d => generated 311 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:30.559267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c8928-2f76-428e-9451-2804ade2d32a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:30.594579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:30.719055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c8928-2f76-428e-9451-2804ade2d32a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:30.758729 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e1ac4-2f9a-494c-ab68-a6d0504128db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:30.759556 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e1ac4-2f9a-494c-ab68-a6d0504128db None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:30.759786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e1ac4-2f9a-494c-ab68-a6d0504128db None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:30.760106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e1ac4-2f9a-494c-ab68-a6d0504128db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:30.845993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e1ac4-2f9a-494c-ab68-a6d0504128db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:30.847788 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e1ac4-2f9a-494c-ab68-a6d0504128db None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:30.880199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:30.973407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 587/1183] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:43:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 14:43:30.980699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c53336-1b13-431e-9d27-2bee6726df4c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:31.111654 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c53336-1b13-431e-9d27-2bee6726df4c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:31.125027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:31.153848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4dad5-1d7d-44b9-a236-56a877aa9151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:31.154507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4dad5-1d7d-44b9-a236-56a877aa9151 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:31.154790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4dad5-1d7d-44b9-a236-56a877aa9151 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:31.155065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4dad5-1d7d-44b9-a236-56a877aa9151 None admin] PATH_INFO: `/v3/projects/7c29e55bc80c4d969f5ed949de24db02` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:31.156353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4dad5-1d7d-44b9-a236-56a877aa9151 None admin] RBAC: Authorizing `identity:get_project(project_id=7c29e55bc80c4d969f5ed949de24db02)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:31.158090 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4dad5-1d7d-44b9-a236-56a877aa9151 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:31.159954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 597/1184] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:43:30 2022] GET /identity/v3/projects/7c29e55bc80c4d969f5ed949de24db02 => generated 361 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:31.166720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e1ac4-2f9a-494c-ab68-a6d0504128db None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:31.298505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e1ac4-2f9a-494c-ab68-a6d0504128db None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:31.347197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:31.374790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc909c6-17b2-4776-af97-7c3f3b3020b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:31.374790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc909c6-17b2-4776-af97-7c3f3b3020b4 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:31.374790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc909c6-17b2-4776-af97-7c3f3b3020b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:31.374790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc909c6-17b2-4776-af97-7c3f3b3020b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:31.375351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc909c6-17b2-4776-af97-7c3f3b3020b4 None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=7c29e55bc80c4d969f5ed949de24db02, user.id=a6a1a89a33d9413bb6773882a965e98d) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:43:31.375351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc909c6-17b2-4776-af97-7c3f3b3020b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:31.375351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc909c6-17b2-4776-af97-7c3f3b3020b4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:31.378790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 588/1185] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:43:31 2022] GET /identity/v3/role_assignments?user.id=a6a1a89a33d9413bb6773882a965e98d&scope.project.id=7c29e55bc80c4d969f5ed949de24db02&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:43:33.560139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449ced3-4d88-42fa-876a-6a24a39066c8 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:33.560744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449ced3-4d88-42fa-876a-6a24a39066c8 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:33.560744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449ced3-4d88-42fa-876a-6a24a39066c8 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:33.562228 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 598/1186] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:33.568443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc98bf5-0fa3-4063-b774-82eccb9746f0 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:33.569309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc98bf5-0fa3-4063-b774-82eccb9746f0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:33.569309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc98bf5-0fa3-4063-b774-82eccb9746f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:33.596932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-4dc98bf5-0fa3-4063-b774-82eccb9746f0 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:33.598460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4dc98bf5-0fa3-4063-b774-82eccb9746f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:33.688922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 589/1187] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:43:33.712900 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449ced3-4d88-42fa-876a-6a24a39066c8 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:33.869690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449ced3-4d88-42fa-876a-6a24a39066c8 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:33.931788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a45282-1a82-4db1-8cca-58c0dfe0ad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:33.932690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45282-1a82-4db1-8cca-58c0dfe0ad99 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:33.933222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45282-1a82-4db1-8cca-58c0dfe0ad99 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:33.933636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45282-1a82-4db1-8cca-58c0dfe0ad99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:33.952975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a45282-1a82-4db1-8cca-58c0dfe0ad99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-75a45282-1a82-4db1-8cca-58c0dfe0ad99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:43:33.961191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:43:33.962266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:43:33.963127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:43:33.963127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:43:33.963127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:43:33.963127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:43:33.963127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:43:33.963127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:43:33.963127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:43:33.963127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:43:33.963127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:43:33.963127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:43:33.963127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:43:33.963127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:43:33.963938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 599/1188] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:43:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:43:33.972583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc98bf5-0fa3-4063-b774-82eccb9746f0 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:34.107175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc98bf5-0fa3-4063-b774-82eccb9746f0 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:34.134296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:34.153053 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ad12e-6a38-46b8-9c69-35a9e0dc0601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:34.153843 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ad12e-6a38-46b8-9c69-35a9e0dc0601 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:34.154247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ad12e-6a38-46b8-9c69-35a9e0dc0601 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:34.154571 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ad12e-6a38-46b8-9c69-35a9e0dc0601 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:34.155414 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ad12e-6a38-46b8-9c69-35a9e0dc0601 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:43:34.155754 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ad12e-6a38-46b8-9c69-35a9e0dc0601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:34.157496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ad12e-6a38-46b8-9c69-35a9e0dc0601 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:34.167961 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 590/1189] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:43:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:34.174086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a45282-1a82-4db1-8cca-58c0dfe0ad99 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:34.223582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:34.317943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a45282-1a82-4db1-8cca-58c0dfe0ad99 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:34.362110 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ea673-3052-4c80-9791-49a47a6e627e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:34.362766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ea673-3052-4c80-9791-49a47a6e627e None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:34.363047 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ea673-3052-4c80-9791-49a47a6e627e None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:34.363322 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ea673-3052-4c80-9791-49a47a6e627e None admin] PATH_INFO: `/v3/users/a6a1a89a33d9413bb6773882a965e98d` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:34.364681 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ea673-3052-4c80-9791-49a47a6e627e None admin] RBAC: Authorizing `identity:get_user(user_id=a6a1a89a33d9413bb6773882a965e98d)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:34.366315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ea673-3052-4c80-9791-49a47a6e627e None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:34.368108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 600/1190] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:43:34 2022] GET /identity/v3/users/a6a1a89a33d9413bb6773882a965e98d => generated 311 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:34.375308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ad12e-6a38-46b8-9c69-35a9e0dc0601 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:34.504866 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ad12e-6a38-46b8-9c69-35a9e0dc0601 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:34.552328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479669f-314d-40a0-b26b-d5401aef7b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:34.552992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479669f-314d-40a0-b26b-d5401aef7b4a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:34.553353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479669f-314d-40a0-b26b-d5401aef7b4a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:34.553659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479669f-314d-40a0-b26b-d5401aef7b4a None admin] PATH_INFO: `/v3/projects/7c29e55bc80c4d969f5ed949de24db02` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:34.555016 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479669f-314d-40a0-b26b-d5401aef7b4a None admin] RBAC: Authorizing `identity:get_project(project_id=7c29e55bc80c4d969f5ed949de24db02)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:34.556509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479669f-314d-40a0-b26b-d5401aef7b4a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:34.558767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 591/1191] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:43:34 2022] GET /identity/v3/projects/7c29e55bc80c4d969f5ed949de24db02 => generated 361 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:43:34.565549 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ea673-3052-4c80-9791-49a47a6e627e None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:34.687655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ea673-3052-4c80-9791-49a47a6e627e None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:34.696483 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:34.727946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e738b-448b-491b-92c8-a7c2f374bea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:34.728631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e738b-448b-491b-92c8-a7c2f374bea4 None admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:34.728908 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e738b-448b-491b-92c8-a7c2f374bea4 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:34.729181 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e738b-448b-491b-92c8-a7c2f374bea4 None admin] PATH_INFO: `/v3/projects/7c29e55bc80c4d969f5ed949de24db02/users/a6a1a89a33d9413bb6773882a965e98d/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:34.732391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e738b-448b-491b-92c8-a7c2f374bea4 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c29e55bc80c4d969f5ed949de24db02, user_id=a6a1a89a33d9413bb6773882a965e98d, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:34.733880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e738b-448b-491b-92c8-a7c2f374bea4 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:34.848845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 601/1192] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:43:34 2022] PUT /identity/v3/projects/7c29e55bc80c4d969f5ed949de24db02/users/a6a1a89a33d9413bb6773882a965e98d/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:43:37.020186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a716ce4-a795-44e8-b157-e94aae980ac6 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:37.021032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a716ce4-a795-44e8-b157-e94aae980ac6 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:37.021032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a716ce4-a795-44e8-b157-e94aae980ac6 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:37.022145 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 592/1193] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:43:37.029301 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a594267-4bc4-4f92-865f-c3ab5241e8f1 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:37.029880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a594267-4bc4-4f92-865f-c3ab5241e8f1 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:37.029880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a594267-4bc4-4f92-865f-c3ab5241e8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:37.051243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-3a594267-4bc4-4f92-865f-c3ab5241e8f1 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:43:37.052572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3a594267-4bc4-4f92-865f-c3ab5241e8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:43:37.131734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 602/1194] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:43:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:43:37.156509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a716ce4-a795-44e8-b157-e94aae980ac6 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:37.221714 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:37.286548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a716ce4-a795-44e8-b157-e94aae980ac6 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:37.328616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed36f7-5184-42e0-b79d-5ed7af0890de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:37.329287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed36f7-5184-42e0-b79d-5ed7af0890de None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:37.329565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed36f7-5184-42e0-b79d-5ed7af0890de None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:37.329840 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed36f7-5184-42e0-b79d-5ed7af0890de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:37.343726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed36f7-5184-42e0-b79d-5ed7af0890de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-88ed36f7-5184-42e0-b79d-5ed7af0890de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:43:37.348865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:43:37.350778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:43:37.352658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:43:37.352658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:43:37.352658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:43:37.352658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:43:37.352658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:43:37.352658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:43:37.352658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:43:37.352658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:43:37.352658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:43:37.352658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:43:37.352658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:43:37.352658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:43:37.352658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 593/1195] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:43:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:43:37.359857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a594267-4bc4-4f92-865f-c3ab5241e8f1 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:37.482376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:37.488753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a594267-4bc4-4f92-865f-c3ab5241e8f1 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:37.529402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb50a58-dbca-4554-997f-bc5532060742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:37.530028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb50a58-dbca-4554-997f-bc5532060742 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:37.530309 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb50a58-dbca-4554-997f-bc5532060742 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:37.530585 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb50a58-dbca-4554-997f-bc5532060742 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:37.531437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb50a58-dbca-4554-997f-bc5532060742 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:43:37.531914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb50a58-dbca-4554-997f-bc5532060742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:37.533392 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb50a58-dbca-4554-997f-bc5532060742 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:37.545315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 603/1196] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:43:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:37.551749 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed36f7-5184-42e0-b79d-5ed7af0890de None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:37.676718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed36f7-5184-42e0-b79d-5ed7af0890de None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:37.690772 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:37.720792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28278b-a1c0-474b-aa3f-195f8a7a6e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:37.721460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28278b-a1c0-474b-aa3f-195f8a7a6e58 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:37.721751 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28278b-a1c0-474b-aa3f-195f8a7a6e58 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:37.722022 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28278b-a1c0-474b-aa3f-195f8a7a6e58 None admin] PATH_INFO: `/v3/users/a6a1a89a33d9413bb6773882a965e98d` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:37.723359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28278b-a1c0-474b-aa3f-195f8a7a6e58 None admin] RBAC: Authorizing `identity:get_user(user_id=a6a1a89a33d9413bb6773882a965e98d)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:37.724865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28278b-a1c0-474b-aa3f-195f8a7a6e58 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:37.726734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 594/1197] 10.222.0.94 () {56 vars in 1177 bytes} [Tue Jul 26 14:43:37 2022] GET /identity/v3/users/a6a1a89a33d9413bb6773882a965e98d => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:43:37.733919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb50a58-dbca-4554-997f-bc5532060742 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:37.915412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb50a58-dbca-4554-997f-bc5532060742 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:37.973762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6aba4-ec46-4266-8bbb-41dd7d419e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:37.974687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6aba4-ec46-4266-8bbb-41dd7d419e9c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:37.975117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6aba4-ec46-4266-8bbb-41dd7d419e9c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:37.976261 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6aba4-ec46-4266-8bbb-41dd7d419e9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:38.023802 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:38.091584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6aba4-ec46-4266-8bbb-41dd7d419e9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:38.094078 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6aba4-ec46-4266-8bbb-41dd7d419e9c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:38.271096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 604/1198] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:43:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:43:38.279096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28278b-a1c0-474b-aa3f-195f8a7a6e58 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:38.410325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28278b-a1c0-474b-aa3f-195f8a7a6e58 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:38.453233 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39124109-a195-408b-b50e-cfa7ce3bb166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:38.453954 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39124109-a195-408b-b50e-cfa7ce3bb166 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:38.454395 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39124109-a195-408b-b50e-cfa7ce3bb166 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:38.454679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39124109-a195-408b-b50e-cfa7ce3bb166 None admin] PATH_INFO: `/v3/projects/7c29e55bc80c4d969f5ed949de24db02` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:38.456102 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39124109-a195-408b-b50e-cfa7ce3bb166 None admin] RBAC: Authorizing `identity:get_project(project_id=7c29e55bc80c4d969f5ed949de24db02)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:38.457538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39124109-a195-408b-b50e-cfa7ce3bb166 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:38.459439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 595/1199] 10.222.0.94 () {56 vars in 1186 bytes} [Tue Jul 26 14:43:38 2022] GET /identity/v3/projects/7c29e55bc80c4d969f5ed949de24db02 => generated 361 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:38.467244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6aba4-ec46-4266-8bbb-41dd7d419e9c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:43:38.549587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:38.600964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6aba4-ec46-4266-8bbb-41dd7d419e9c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:43:38.642494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8b032-3a91-465d-a273-07c901bd1e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:43:38.643251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8b032-3a91-465d-a273-07c901bd1e4d None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:43:38.643666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8b032-3a91-465d-a273-07c901bd1e4d None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:43:38.644036 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8b032-3a91-465d-a273-07c901bd1e4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:43:38.644926 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8b032-3a91-465d-a273-07c901bd1e4d None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=7c29e55bc80c4d969f5ed949de24db02, user.id=a6a1a89a33d9413bb6773882a965e98d) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:43:38.645320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8b032-3a91-465d-a273-07c901bd1e4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:43:38.646942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8b032-3a91-465d-a273-07c901bd1e4d None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:43:38.657823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 605/1200] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:43:38 2022] GET /identity/v3/role_assignments?user.id=a6a1a89a33d9413bb6773882a965e98d&scope.project.id=7c29e55bc80c4d969f5ed949de24db02&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:43:48.447705 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=77 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:43:48.651915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=39 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:03.916085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f2c39-09ee-422d-a6dd-e90080a92ef9 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:03.916818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f2c39-09ee-422d-a6dd-e90080a92ef9 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:03.916818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f2c39-09ee-422d-a6dd-e90080a92ef9 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:03.918019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 596/1201] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:03.924784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50588ee4-fa4e-4cb1-9098-fc90d1d5d51b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:03.927433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50588ee4-fa4e-4cb1-9098-fc90d1d5d51b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:03.927433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50588ee4-fa4e-4cb1-9098-fc90d1d5d51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:03.952507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-50588ee4-fa4e-4cb1-9098-fc90d1d5d51b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:03.953743 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-50588ee4-fa4e-4cb1-9098-fc90d1d5d51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:04.047571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 606/1202] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:45:04.076934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f2c39-09ee-422d-a6dd-e90080a92ef9 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:04.205230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f2c39-09ee-422d-a6dd-e90080a92ef9 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:04.243297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba6fdd-b3fb-4749-9704-3148987300cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:04.244001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba6fdd-b3fb-4749-9704-3148987300cd None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:04.244289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba6fdd-b3fb-4749-9704-3148987300cd None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:04.244561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba6fdd-b3fb-4749-9704-3148987300cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:04.256908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba6fdd-b3fb-4749-9704-3148987300cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:04.259098 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba6fdd-b3fb-4749-9704-3148987300cd None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-22ba6fdd-b3fb-4749-9704-3148987300cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:45:04.276002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:04.277271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:04.278334 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:45:04.278334 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:45:04.278334 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:04.278334 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:04.278334 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 597/1203] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:45:04.284525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50588ee4-fa4e-4cb1-9098-fc90d1d5d51b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:04.409949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50588ee4-fa4e-4cb1-9098-fc90d1d5d51b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:04.443778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:04.450972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edea100-cc39-4c74-bc53-94f03587b3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:04.451633 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edea100-cc39-4c74-bc53-94f03587b3e4 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:04.451953 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edea100-cc39-4c74-bc53-94f03587b3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:04.452230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edea100-cc39-4c74-bc53-94f03587b3e4 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:04.452926 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edea100-cc39-4c74-bc53-94f03587b3e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:04.453235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edea100-cc39-4c74-bc53-94f03587b3e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:04.454675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edea100-cc39-4c74-bc53-94f03587b3e4 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:04.470138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 607/1204] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:04.476301 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba6fdd-b3fb-4749-9704-3148987300cd None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:04.541012 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:04.602136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba6fdd-b3fb-4749-9704-3148987300cd None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:04.642086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5135d7-fc41-4fb2-941f-d49c519a70b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:04.642713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5135d7-fc41-4fb2-941f-d49c519a70b8 None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:04.642982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5135d7-fc41-4fb2-941f-d49c519a70b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:04.643252 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5135d7-fc41-4fb2-941f-d49c519a70b8 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:04.643965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5135d7-fc41-4fb2-941f-d49c519a70b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:04.645382 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5135d7-fc41-4fb2-941f-d49c519a70b8 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:04.770397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 598/1205] 10.222.0.94 () {60 vars in 1132 bytes} [Tue Jul 26 14:45:04 2022] POST /identity/v3/users => generated 251 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:45:06.973859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ce5ec-ee56-42ee-882d-500f5e2419c0 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:06.974570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ce5ec-ee56-42ee-882d-500f5e2419c0 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:06.974570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ce5ec-ee56-42ee-882d-500f5e2419c0 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:06.976192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 608/1206] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:45:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:45:06.982696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c358c-dfcd-40b5-acf7-338274676e72 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:06.983611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c358c-dfcd-40b5-acf7-338274676e72 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:06.983794 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c358c-dfcd-40b5-acf7-338274676e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:07.007694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-925c358c-dfcd-40b5-acf7-338274676e72 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:07.009167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-925c358c-dfcd-40b5-acf7-338274676e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:07.095234 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 599/1207] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:45:07.121221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ce5ec-ee56-42ee-882d-500f5e2419c0 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:07.279301 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ce5ec-ee56-42ee-882d-500f5e2419c0 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:07.328776 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17786a-5645-4856-9ed6-503c31399e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:07.329560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17786a-5645-4856-9ed6-503c31399e18 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:07.329883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17786a-5645-4856-9ed6-503c31399e18 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:07.330187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17786a-5645-4856-9ed6-503c31399e18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:07.345515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17786a-5645-4856-9ed6-503c31399e18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-fb17786a-5645-4856-9ed6-503c31399e18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:45:07.351236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:07.353049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:07.354888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:07.354888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:07.354888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:07.354888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:07.354888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:45:07.354888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:45:07.354888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:45:07.354888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:45:07.354888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:45:07.354888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:45:07.354888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:45:07.354888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:07.354888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 609/1208] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:45:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:07.361006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c358c-dfcd-40b5-acf7-338274676e72 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:07.412370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:07.514831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c358c-dfcd-40b5-acf7-338274676e72 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:07.584587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13a429-1b8b-4b98-8f3a-685c04e001fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:07.586522 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13a429-1b8b-4b98-8f3a-685c04e001fc None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:07.587026 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13a429-1b8b-4b98-8f3a-685c04e001fc None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:07.587548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13a429-1b8b-4b98-8f3a-685c04e001fc None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:07.589017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13a429-1b8b-4b98-8f3a-685c04e001fc None admin] RBAC: Adding query filter params (name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:07.589579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13a429-1b8b-4b98-8f3a-685c04e001fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:07.592419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13a429-1b8b-4b98-8f3a-685c04e001fc None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:07.608435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 600/1209] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:45:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:07.618702 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17786a-5645-4856-9ed6-503c31399e18 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:07.718165 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:07.780228 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17786a-5645-4856-9ed6-503c31399e18 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:07.831002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd5198-e8a1-4ee3-a1eb-86bd7a9b5775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:07.832135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5198-e8a1-4ee3-a1eb-86bd7a9b5775 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:07.832642 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5198-e8a1-4ee3-a1eb-86bd7a9b5775 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:07.833108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5198-e8a1-4ee3-a1eb-86bd7a9b5775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:07.856695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd5198-e8a1-4ee3-a1eb-86bd7a9b5775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:07.859537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd5198-e8a1-4ee3-a1eb-86bd7a9b5775 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-4ccd5198-e8a1-4ee3-a1eb-86bd7a9b5775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:45:07.888307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:07.890984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:07.892478 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:45:07.892478 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:45:07.892478 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:07.892478 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:07.892478 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 610/1210] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:07.896550 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13a429-1b8b-4b98-8f3a-685c04e001fc None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:08.025777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:08.029625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13a429-1b8b-4b98-8f3a-685c04e001fc None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:08.079691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fd9a0-0186-4674-98cd-9d5b79ff76f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:08.080639 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fd9a0-0186-4674-98cd-9d5b79ff76f1 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:08.081100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fd9a0-0186-4674-98cd-9d5b79ff76f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:08.081620 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fd9a0-0186-4674-98cd-9d5b79ff76f1 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:08.082846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fd9a0-0186-4674-98cd-9d5b79ff76f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:08.083359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fd9a0-0186-4674-98cd-9d5b79ff76f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:08.085896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fd9a0-0186-4674-98cd-9d5b79ff76f1 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:08.111734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 601/1211] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:08.119000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd5198-e8a1-4ee3-a1eb-86bd7a9b5775 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:08.289450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd5198-e8a1-4ee3-a1eb-86bd7a9b5775 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:08.291790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:08.342891 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a3a72-c3dd-4875-ad43-132bbdb9f727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:08.343530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a3a72-c3dd-4875-ad43-132bbdb9f727 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:08.343869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a3a72-c3dd-4875-ad43-132bbdb9f727 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:08.344181 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a3a72-c3dd-4875-ad43-132bbdb9f727 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:08.354343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a3a72-c3dd-4875-ad43-132bbdb9f727 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:08.357275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a3a72-c3dd-4875-ad43-132bbdb9f727 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-a24a3a72-c3dd-4875-ad43-132bbdb9f727 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:45:08.379295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:08.380676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:08.382459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:08.382459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:08.382459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:45:08.382459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:45:08.382459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:45:08.382459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:45:08.382459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:45:08.382459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:45:08.382459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:45:08.382459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:08.382459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 611/1212] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:08.392444 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fd9a0-0186-4674-98cd-9d5b79ff76f1 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:08.576459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fd9a0-0186-4674-98cd-9d5b79ff76f1 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:08.622149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab70030-0c11-4d1e-b759-feb2c17748d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:08.622673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab70030-0c11-4d1e-b759-feb2c17748d2 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:08.622976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab70030-0c11-4d1e-b759-feb2c17748d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:08.623293 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab70030-0c11-4d1e-b759-feb2c17748d2 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:08.624182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab70030-0c11-4d1e-b759-feb2c17748d2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:08.624486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab70030-0c11-4d1e-b759-feb2c17748d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:08.626004 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab70030-0c11-4d1e-b759-feb2c17748d2 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:08.649573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 602/1213] 10.222.0.94 () {56 vars in 1143 bytes} [Tue Jul 26 14:45:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:08.660370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a3a72-c3dd-4875-ad43-132bbdb9f727 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:08.877078 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a3a72-c3dd-4875-ad43-132bbdb9f727 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:08.929435 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c1717-cb2c-4d0d-aa81-77d24a5bd7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:08.930520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c1717-cb2c-4d0d-aa81-77d24a5bd7ec None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:08.931546 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c1717-cb2c-4d0d-aa81-77d24a5bd7ec None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:08.935390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c1717-cb2c-4d0d-aa81-77d24a5bd7ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:08.997676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:09.072948 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c1717-cb2c-4d0d-aa81-77d24a5bd7ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:09.075261 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c1717-cb2c-4d0d-aa81-77d24a5bd7ec None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:09.223160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 612/1214] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:45:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:45:09.230461 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab70030-0c11-4d1e-b759-feb2c17748d2 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:09.266350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:09.372071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab70030-0c11-4d1e-b759-feb2c17748d2 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:09.420734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e008aa-42bc-4da0-a949-4f69409ca8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:09.421452 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e008aa-42bc-4da0-a949-4f69409ca8ba None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:09.421811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e008aa-42bc-4da0-a949-4f69409ca8ba None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:09.422251 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e008aa-42bc-4da0-a949-4f69409ca8ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:09.435372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e008aa-42bc-4da0-a949-4f69409ca8ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:09.437846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e008aa-42bc-4da0-a949-4f69409ca8ba None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-f6e008aa-42bc-4da0-a949-4f69409ca8ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:45:09.457958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:09.464460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:09.466139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:45:09.466139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:45:09.466139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:09.466139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:09.466139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 603/1215] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:09.468592 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c1717-cb2c-4d0d-aa81-77d24a5bd7ec None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:09.560340 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:09.610857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c1717-cb2c-4d0d-aa81-77d24a5bd7ec None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:09.656921 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33437b84-5662-4140-8839-700e456b4fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:09.657542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33437b84-5662-4140-8839-700e456b4fed None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:09.657819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33437b84-5662-4140-8839-700e456b4fed None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:09.658083 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33437b84-5662-4140-8839-700e456b4fed None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:09.658802 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33437b84-5662-4140-8839-700e456b4fed None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:09.659133 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33437b84-5662-4140-8839-700e456b4fed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:09.660505 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33437b84-5662-4140-8839-700e456b4fed None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:09.674061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 613/1216] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:09.680352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e008aa-42bc-4da0-a949-4f69409ca8ba None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:09.773472 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:09.811261 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e008aa-42bc-4da0-a949-4f69409ca8ba None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:09.855054 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253b531-5a46-4df5-b7a2-3f59b4219a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:09.855754 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253b531-5a46-4df5-b7a2-3f59b4219a9c None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:09.856135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253b531-5a46-4df5-b7a2-3f59b4219a9c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:09.856439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253b531-5a46-4df5-b7a2-3f59b4219a9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:09.864686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253b531-5a46-4df5-b7a2-3f59b4219a9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:09.866297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253b531-5a46-4df5-b7a2-3f59b4219a9c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-7253b531-5a46-4df5-b7a2-3f59b4219a9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:45:09.880837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:09.882776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:09.885057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:45:09.885057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:45:09.885057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:45:09.885057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:45:09.885057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:45:09.885057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:45:09.885057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:45:09.885057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:09.885057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 604/1217] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:45:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:09.890987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33437b84-5662-4140-8839-700e456b4fed None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:10.062404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33437b84-5662-4140-8839-700e456b4fed None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:10.108153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:10.129799 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66157bab-3168-43b6-b51d-092328f465d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:10.130832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66157bab-3168-43b6-b51d-092328f465d5 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:10.131315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66157bab-3168-43b6-b51d-092328f465d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:10.131905 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66157bab-3168-43b6-b51d-092328f465d5 None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:10.133317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66157bab-3168-43b6-b51d-092328f465d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:10.133807 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66157bab-3168-43b6-b51d-092328f465d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:10.136681 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66157bab-3168-43b6-b51d-092328f465d5 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:10.157575 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 614/1218] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:45:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:10.164630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253b531-5a46-4df5-b7a2-3f59b4219a9c None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:10.296264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253b531-5a46-4df5-b7a2-3f59b4219a9c None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:10.339859 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:10.346638 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a022fa-76ed-42a5-b511-40989e5c8103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:10.347292 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a022fa-76ed-42a5-b511-40989e5c8103 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:10.347604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a022fa-76ed-42a5-b511-40989e5c8103 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:10.347878 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a022fa-76ed-42a5-b511-40989e5c8103 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:10.348696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a022fa-76ed-42a5-b511-40989e5c8103 None admin] RBAC: Adding query filter params (role.id=170d72cc35f34a249c65a819a18638a4, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=42a0728e2f6249bdb45d6f44fb15aa4c) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:10.349003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a022fa-76ed-42a5-b511-40989e5c8103 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:10.351000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a022fa-76ed-42a5-b511-40989e5c8103 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:10.358443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 605/1219] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:45:10 2022] GET /identity/v3/role_assignments?user.id=42a0728e2f6249bdb45d6f44fb15aa4c&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=170d72cc35f34a249c65a819a18638a4 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:45:12.627911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d46ad2-32db-4f43-b359-a9b8de32dfde None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:12.628477 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d46ad2-32db-4f43-b359-a9b8de32dfde None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:12.628477 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d46ad2-32db-4f43-b359-a9b8de32dfde None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:12.629808 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 615/1220] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:12.635820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652ae99-1515-45f0-a087-ca6c44f43386 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:12.636610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652ae99-1515-45f0-a087-ca6c44f43386 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:12.636766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652ae99-1515-45f0-a087-ca6c44f43386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:12.657785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-1652ae99-1515-45f0-a087-ca6c44f43386 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:12.659050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1652ae99-1515-45f0-a087-ca6c44f43386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:12.737524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 606/1221] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:45:12.762667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d46ad2-32db-4f43-b359-a9b8de32dfde None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:12.895305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d46ad2-32db-4f43-b359-a9b8de32dfde None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:12.955607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe6a50-e5de-4279-945b-f73801b54440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:12.956586 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe6a50-e5de-4279-945b-f73801b54440 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:12.956999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe6a50-e5de-4279-945b-f73801b54440 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:12.957380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe6a50-e5de-4279-945b-f73801b54440 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:12.972031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe6a50-e5de-4279-945b-f73801b54440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-93fe6a50-e5de-4279-945b-f73801b54440 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:45:12.978333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:12.980431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:12.982363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:12.982363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:12.982363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:12.982363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:12.982363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:45:12.982363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:45:12.982363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:45:12.982363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:45:12.982363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:45:12.982363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:45:12.982363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:45:12.982363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:12.982363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 616/1222] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:45:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:12.987832 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652ae99-1515-45f0-a087-ca6c44f43386 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:13.114343 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652ae99-1515-45f0-a087-ca6c44f43386 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:13.163351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:13.182256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df30899-c406-41c8-b43f-4c0106ec4d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:13.183269 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df30899-c406-41c8-b43f-4c0106ec4d36 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:13.183799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df30899-c406-41c8-b43f-4c0106ec4d36 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:13.184283 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df30899-c406-41c8-b43f-4c0106ec4d36 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:13.185671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df30899-c406-41c8-b43f-4c0106ec4d36 None admin] RBAC: Adding query filter params (name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:13.186205 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df30899-c406-41c8-b43f-4c0106ec4d36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:13.188890 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df30899-c406-41c8-b43f-4c0106ec4d36 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:13.203530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 607/1223] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:45:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:45:13.213723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe6a50-e5de-4279-945b-f73801b54440 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:13.288733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:13.362918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe6a50-e5de-4279-945b-f73801b54440 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:13.406884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa41f3f-cac8-4efa-9f85-0644b8bfd96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:13.407685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa41f3f-cac8-4efa-9f85-0644b8bfd96c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:13.408037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa41f3f-cac8-4efa-9f85-0644b8bfd96c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:13.408166 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa41f3f-cac8-4efa-9f85-0644b8bfd96c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:13.421262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa41f3f-cac8-4efa-9f85-0644b8bfd96c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:13.422645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa41f3f-cac8-4efa-9f85-0644b8bfd96c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-1aa41f3f-cac8-4efa-9f85-0644b8bfd96c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:45:13.440283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:13.442010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:13.443787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:45:13.443787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:45:13.443787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:13.443787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:13.443787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 617/1224] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:45:13.450031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df30899-c406-41c8-b43f-4c0106ec4d36 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:13.577628 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df30899-c406-41c8-b43f-4c0106ec4d36 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:13.626261 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502b978-de35-46a1-8f00-c7ce925b940d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:13.626900 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502b978-de35-46a1-8f00-c7ce925b940d None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:13.627186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502b978-de35-46a1-8f00-c7ce925b940d None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:13.627506 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502b978-de35-46a1-8f00-c7ce925b940d None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:13.628279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502b978-de35-46a1-8f00-c7ce925b940d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:13.628593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502b978-de35-46a1-8f00-c7ce925b940d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:13.630138 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502b978-de35-46a1-8f00-c7ce925b940d None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:13.646037 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 608/1225] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:13.653171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa41f3f-cac8-4efa-9f85-0644b8bfd96c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:13.792890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:13.800745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa41f3f-cac8-4efa-9f85-0644b8bfd96c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:13.846582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337433ca-bb7a-4138-a9e7-6986533748ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:13.847405 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337433ca-bb7a-4138-a9e7-6986533748ac None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:13.847759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337433ca-bb7a-4138-a9e7-6986533748ac None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:13.848145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337433ca-bb7a-4138-a9e7-6986533748ac None admin] PATH_INFO: `/v3/users/placement` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:13.858001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337433ca-bb7a-4138-a9e7-6986533748ac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:13.859578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337433ca-bb7a-4138-a9e7-6986533748ac None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-337433ca-bb7a-4138-a9e7-6986533748ac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:45:13.873964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:13.875978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:13.877423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:13.877423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:13.877423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:45:13.877423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:45:13.877423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:45:13.877423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:45:13.877423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:45:13.877423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:45:13.877423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:45:13.877423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:13.877423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 618/1226] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:13.885669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502b978-de35-46a1-8f00-c7ce925b940d None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:13.945349 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:14.020161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502b978-de35-46a1-8f00-c7ce925b940d None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:14.065010 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962ee7e-50fb-449f-bfb3-3ce510d3986a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:14.065548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962ee7e-50fb-449f-bfb3-3ce510d3986a None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:14.065830 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962ee7e-50fb-449f-bfb3-3ce510d3986a None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:14.066112 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962ee7e-50fb-449f-bfb3-3ce510d3986a None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:14.066944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962ee7e-50fb-449f-bfb3-3ce510d3986a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:14.067279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962ee7e-50fb-449f-bfb3-3ce510d3986a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:14.069142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962ee7e-50fb-449f-bfb3-3ce510d3986a None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:14.092400 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 609/1227] 10.222.0.94 () {56 vars in 1143 bytes} [Tue Jul 26 14:45:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:14.099421 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337433ca-bb7a-4138-a9e7-6986533748ac None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:14.230127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337433ca-bb7a-4138-a9e7-6986533748ac None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:14.275721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd64cb-6ce9-4f1a-997b-b3b2309c9a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:14.276428 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd64cb-6ce9-4f1a-997b-b3b2309c9a1f None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:14.276725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd64cb-6ce9-4f1a-997b-b3b2309c9a1f None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:14.277038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd64cb-6ce9-4f1a-997b-b3b2309c9a1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:14.291722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd64cb-6ce9-4f1a-997b-b3b2309c9a1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:14.293331 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd64cb-6ce9-4f1a-997b-b3b2309c9a1f None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-5efd64cb-6ce9-4f1a-997b-b3b2309c9a1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:45:14.319050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:14.320220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:14.321410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:45:14.321410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:45:14.321410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:14.321410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:14.321410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 619/1228] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:14.328270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962ee7e-50fb-449f-bfb3-3ce510d3986a None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:14.456345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962ee7e-50fb-449f-bfb3-3ce510d3986a None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:14.459158 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:14.495674 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ac86c-8a09-4c3d-8e1e-58d984aee808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:14.498576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ac86c-8a09-4c3d-8e1e-58d984aee808 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:14.498576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ac86c-8a09-4c3d-8e1e-58d984aee808 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:14.498576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ac86c-8a09-4c3d-8e1e-58d984aee808 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:14.498576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ac86c-8a09-4c3d-8e1e-58d984aee808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:14.498576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ac86c-8a09-4c3d-8e1e-58d984aee808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:14.499319 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ac86c-8a09-4c3d-8e1e-58d984aee808 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:14.513441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 610/1229] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:14.520131 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd64cb-6ce9-4f1a-997b-b3b2309c9a1f None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:14.531275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:14.645456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd64cb-6ce9-4f1a-997b-b3b2309c9a1f None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:14.685804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa048c-4bb6-41dc-ba94-30d45d556615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:14.695407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa048c-4bb6-41dc-ba94-30d45d556615 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:14.695407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa048c-4bb6-41dc-ba94-30d45d556615 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:14.695407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa048c-4bb6-41dc-ba94-30d45d556615 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:14.695407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa048c-4bb6-41dc-ba94-30d45d556615 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:14.696221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa048c-4bb6-41dc-ba94-30d45d556615 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-a5fa048c-4bb6-41dc-ba94-30d45d556615 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:45:14.708015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:14.709117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:14.710986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:45:14.710986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:45:14.710986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:45:14.710986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:45:14.710986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:45:14.710986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:45:14.710986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:45:14.710986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:14.710986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 620/1230] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:45:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:14.716153 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ac86c-8a09-4c3d-8e1e-58d984aee808 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:14.840684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ac86c-8a09-4c3d-8e1e-58d984aee808 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:14.897572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a0f68-ad15-4c72-a4c8-323896758e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:14.898238 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a0f68-ad15-4c72-a4c8-323896758e62 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:14.898571 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a0f68-ad15-4c72-a4c8-323896758e62 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:14.898952 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a0f68-ad15-4c72-a4c8-323896758e62 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:14.900047 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a0f68-ad15-4c72-a4c8-323896758e62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:14.900372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a0f68-ad15-4c72-a4c8-323896758e62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:14.902288 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a0f68-ad15-4c72-a4c8-323896758e62 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:14.918828 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 611/1231] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:45:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:14.925125 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa048c-4bb6-41dc-ba94-30d45d556615 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:14.995735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:15.034297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa048c-4bb6-41dc-ba94-30d45d556615 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:15.075753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f95ddc-d62e-49f3-af79-119772dbfb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:15.076431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f95ddc-d62e-49f3-af79-119772dbfb11 None admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:15.076716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f95ddc-d62e-49f3-af79-119772dbfb11 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:15.076991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f95ddc-d62e-49f3-af79-119772dbfb11 None admin] PATH_INFO: `/v3/projects/a9b328041da24cedbc801b271d6e0419/users/42a0728e2f6249bdb45d6f44fb15aa4c/roles/170d72cc35f34a249c65a819a18638a4` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:15.094045 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f95ddc-d62e-49f3-af79-119772dbfb11 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9b328041da24cedbc801b271d6e0419, user_id=42a0728e2f6249bdb45d6f44fb15aa4c, role_id=170d72cc35f34a249c65a819a18638a4)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:15.095512 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f95ddc-d62e-49f3-af79-119772dbfb11 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:15.178850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 621/1232] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:45:14 2022] PUT /identity/v3/projects/a9b328041da24cedbc801b271d6e0419/users/42a0728e2f6249bdb45d6f44fb15aa4c/roles/170d72cc35f34a249c65a819a18638a4 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:45:17.276569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a08d9c-f8a9-4130-a946-a5af8626b42f None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:17.277123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a08d9c-f8a9-4130-a946-a5af8626b42f None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:17.277123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a08d9c-f8a9-4130-a946-a5af8626b42f None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:17.278476 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 612/1233] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:17.284591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9ca6f-d9b0-428a-bb0d-f5cd4dbc77a9 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:17.285124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9ca6f-d9b0-428a-bb0d-f5cd4dbc77a9 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:17.285124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9ca6f-d9b0-428a-bb0d-f5cd4dbc77a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:17.303837 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7db9ca6f-d9b0-428a-bb0d-f5cd4dbc77a9 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:17.305183 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7db9ca6f-d9b0-428a-bb0d-f5cd4dbc77a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:17.377406 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 622/1234] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:45:17.402830 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a08d9c-f8a9-4130-a946-a5af8626b42f None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:17.431076 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:17.532071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a08d9c-f8a9-4130-a946-a5af8626b42f None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:17.572849 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9006e-4f1b-4ef0-a324-d51f8dba42c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:17.573492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9006e-4f1b-4ef0-a324-d51f8dba42c6 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:17.573774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9006e-4f1b-4ef0-a324-d51f8dba42c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:17.574050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9006e-4f1b-4ef0-a324-d51f8dba42c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:17.584984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9006e-4f1b-4ef0-a324-d51f8dba42c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-65d9006e-4f1b-4ef0-a324-d51f8dba42c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:45:17.589975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:17.591135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:17.597359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:17.597359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:17.597359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:17.597359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:17.597359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:45:17.597359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:45:17.597359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:45:17.597359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:45:17.597359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:45:17.597359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:45:17.597359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 14:45:17.597359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:17.597359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 613/1235] 10.222.0.94 () {56 vars in 1102 bytes} [Tue Jul 26 14:45:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:17.598077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9ca6f-d9b0-428a-bb0d-f5cd4dbc77a9 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:17.658667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:17.723371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9ca6f-d9b0-428a-bb0d-f5cd4dbc77a9 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:17.765580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8e38d-60ad-4427-af67-6bc346be3690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:17.766202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8e38d-60ad-4427-af67-6bc346be3690 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:17.766473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8e38d-60ad-4427-af67-6bc346be3690 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:17.766742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8e38d-60ad-4427-af67-6bc346be3690 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:17.767515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8e38d-60ad-4427-af67-6bc346be3690 None admin] RBAC: Adding query filter params (name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:17.767812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8e38d-60ad-4427-af67-6bc346be3690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:17.769180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8e38d-60ad-4427-af67-6bc346be3690 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:17.779320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 623/1236] 10.222.0.94 () {56 vars in 1103 bytes} [Tue Jul 26 14:45:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:45:17.785825 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9006e-4f1b-4ef0-a324-d51f8dba42c6 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:17.882634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:17.901438 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9006e-4f1b-4ef0-a324-d51f8dba42c6 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:17.936860 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b68e7-de11-471d-a3ac-d2427d865abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:17.937600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b68e7-de11-471d-a3ac-d2427d865abe None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:17.937881 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b68e7-de11-471d-a3ac-d2427d865abe None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:17.938158 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b68e7-de11-471d-a3ac-d2427d865abe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:17.951277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b68e7-de11-471d-a3ac-d2427d865abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:17.952818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b68e7-de11-471d-a3ac-d2427d865abe None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-931b68e7-de11-471d-a3ac-d2427d865abe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:45:17.968920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:17.970176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:17.971088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:45:17.971088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:45:17.971088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:17.971088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:17.971088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 614/1237] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:17.976112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8e38d-60ad-4427-af67-6bc346be3690 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:18.088934 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8e38d-60ad-4427-af67-6bc346be3690 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:18.098391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:18.125454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a820081-3b12-44b3-a5b1-e6d672b72b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:18.126104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a820081-3b12-44b3-a5b1-e6d672b72b36 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:18.126462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a820081-3b12-44b3-a5b1-e6d672b72b36 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:18.126656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a820081-3b12-44b3-a5b1-e6d672b72b36 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:18.127437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a820081-3b12-44b3-a5b1-e6d672b72b36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:18.127774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a820081-3b12-44b3-a5b1-e6d672b72b36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:18.129151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a820081-3b12-44b3-a5b1-e6d672b72b36 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:18.148141 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 624/1238] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:18.155175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b68e7-de11-471d-a3ac-d2427d865abe None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:18.282406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b68e7-de11-471d-a3ac-d2427d865abe None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:18.324356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119689a-fd56-4ed2-b2a5-df5d11b3a841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:18.325006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119689a-fd56-4ed2-b2a5-df5d11b3a841 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:18.325289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119689a-fd56-4ed2-b2a5-df5d11b3a841 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:18.325564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119689a-fd56-4ed2-b2a5-df5d11b3a841 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:18.328567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:18.337650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119689a-fd56-4ed2-b2a5-df5d11b3a841 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:18.340615 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119689a-fd56-4ed2-b2a5-df5d11b3a841 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-1119689a-fd56-4ed2-b2a5-df5d11b3a841 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:45:18.361464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:18.362589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:18.363734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:18.363734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:18.363734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:45:18.363734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:45:18.363734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:45:18.363734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:45:18.363734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:45:18.363734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:45:18.363734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:45:18.363734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:18.363734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 615/1239] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:18.370144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a820081-3b12-44b3-a5b1-e6d672b72b36 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:18.479778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a820081-3b12-44b3-a5b1-e6d672b72b36 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:18.527369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f96294-ce9f-4da7-a2d1-6c2825dfdb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:18.528041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f96294-ce9f-4da7-a2d1-6c2825dfdb0c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:18.528321 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f96294-ce9f-4da7-a2d1-6c2825dfdb0c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:18.528598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f96294-ce9f-4da7-a2d1-6c2825dfdb0c None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:18.529690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f96294-ce9f-4da7-a2d1-6c2825dfdb0c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:18.529998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f96294-ce9f-4da7-a2d1-6c2825dfdb0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:18.531426 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f96294-ce9f-4da7-a2d1-6c2825dfdb0c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:18.551772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 625/1240] 10.222.0.94 () {56 vars in 1143 bytes} [Tue Jul 26 14:45:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:18.558998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119689a-fd56-4ed2-b2a5-df5d11b3a841 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:18.682570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119689a-fd56-4ed2-b2a5-df5d11b3a841 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:18.734671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ebc25-d8e0-4f14-b50d-e453b16caa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:18.735564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ebc25-d8e0-4f14-b50d-e453b16caa97 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:18.735981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ebc25-d8e0-4f14-b50d-e453b16caa97 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:18.736385 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ebc25-d8e0-4f14-b50d-e453b16caa97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:18.833986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:18.843186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ebc25-d8e0-4f14-b50d-e453b16caa97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:18.844708 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ebc25-d8e0-4f14-b50d-e453b16caa97 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:18.952146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 616/1241] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:45:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:45:18.958267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f96294-ce9f-4da7-a2d1-6c2825dfdb0c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:18.988299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:19.070478 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f96294-ce9f-4da7-a2d1-6c2825dfdb0c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:19.107119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899120d6-4451-42fb-b196-8c8c2dc69988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:19.107815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899120d6-4451-42fb-b196-8c8c2dc69988 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:19.108089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899120d6-4451-42fb-b196-8c8c2dc69988 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:19.108359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899120d6-4451-42fb-b196-8c8c2dc69988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:19.123801 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899120d6-4451-42fb-b196-8c8c2dc69988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:19.126394 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899120d6-4451-42fb-b196-8c8c2dc69988 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-899120d6-4451-42fb-b196-8c8c2dc69988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:45:19.155761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:19.156942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:19.158041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:45:19.158041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:45:19.158041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:19.158041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:19.158289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 626/1242] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:19.164288 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ebc25-d8e0-4f14-b50d-e453b16caa97 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:19.269973 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:19.276746 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ebc25-d8e0-4f14-b50d-e453b16caa97 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:19.319272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a97a0-ba03-42ee-a4cf-889f8dbe57b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:19.320706 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a97a0-ba03-42ee-a4cf-889f8dbe57b3 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:19.321200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a97a0-ba03-42ee-a4cf-889f8dbe57b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:19.321632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a97a0-ba03-42ee-a4cf-889f8dbe57b3 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:19.322880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a97a0-ba03-42ee-a4cf-889f8dbe57b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:19.323439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a97a0-ba03-42ee-a4cf-889f8dbe57b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:19.325298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a97a0-ba03-42ee-a4cf-889f8dbe57b3 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:19.343089 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 617/1243] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:45:19.350037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899120d6-4451-42fb-b196-8c8c2dc69988 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:19.442817 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:19.464478 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899120d6-4451-42fb-b196-8c8c2dc69988 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:19.501296 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e72a7-5080-4608-8798-2f5b55fa6a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:19.501863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e72a7-5080-4608-8798-2f5b55fa6a86 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:19.502145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e72a7-5080-4608-8798-2f5b55fa6a86 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:19.502522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e72a7-5080-4608-8798-2f5b55fa6a86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:19.509757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e72a7-5080-4608-8798-2f5b55fa6a86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:19.511219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e72a7-5080-4608-8798-2f5b55fa6a86 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-a04e72a7-5080-4608-8798-2f5b55fa6a86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:45:19.523124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:19.524303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:19.525535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:45:19.525535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:45:19.525535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:45:19.525535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:45:19.525535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:45:19.525535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:45:19.525535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:45:19.525535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:19.525535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 627/1244] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:45:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:19.530651 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a97a0-ba03-42ee-a4cf-889f8dbe57b3 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:19.650279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a97a0-ba03-42ee-a4cf-889f8dbe57b3 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:19.688190 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfccd2-1934-4b6b-8c97-ddced200779f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:19.688960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfccd2-1934-4b6b-8c97-ddced200779f None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:19.689290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfccd2-1934-4b6b-8c97-ddced200779f None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:19.689615 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfccd2-1934-4b6b-8c97-ddced200779f None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:19.690471 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfccd2-1934-4b6b-8c97-ddced200779f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:19.690775 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfccd2-1934-4b6b-8c97-ddced200779f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:19.692235 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfccd2-1934-4b6b-8c97-ddced200779f None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:19.713256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 618/1245] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:45:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:19.719800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e72a7-5080-4608-8798-2f5b55fa6a86 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:19.847223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e72a7-5080-4608-8798-2f5b55fa6a86 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:19.887077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1e2f5-e192-4de0-b322-2e7bd62a991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:19.888065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1e2f5-e192-4de0-b322-2e7bd62a991a None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:19.888348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1e2f5-e192-4de0-b322-2e7bd62a991a None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:19.888734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1e2f5-e192-4de0-b322-2e7bd62a991a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:19.889855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1e2f5-e192-4de0-b322-2e7bd62a991a None admin] RBAC: Adding query filter params (role.id=170d72cc35f34a249c65a819a18638a4, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=42a0728e2f6249bdb45d6f44fb15aa4c) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:19.890291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1e2f5-e192-4de0-b322-2e7bd62a991a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:19.891993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1e2f5-e192-4de0-b322-2e7bd62a991a None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:19.899088 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 628/1246] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:45:19 2022] GET /identity/v3/role_assignments?user.id=42a0728e2f6249bdb45d6f44fb15aa4c&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=170d72cc35f34a249c65a819a18638a4 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:22.114248 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff28c42-ecfb-443d-a6f8-efcaf7e42c76 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:22.115068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff28c42-ecfb-443d-a6f8-efcaf7e42c76 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:22.115068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff28c42-ecfb-443d-a6f8-efcaf7e42c76 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:22.117085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 619/1247] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:45:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:22.123332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef185d0a-9ed0-4bb5-b5b2-894fb8e4b0cf None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:22.124196 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef185d0a-9ed0-4bb5-b5b2-894fb8e4b0cf None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:22.124196 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef185d0a-9ed0-4bb5-b5b2-894fb8e4b0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:22.131006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:22.151373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-ef185d0a-9ed0-4bb5-b5b2-894fb8e4b0cf None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:22.153305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ef185d0a-9ed0-4bb5-b5b2-894fb8e4b0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:22.248988 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 629/1248] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:45:22.273798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff28c42-ecfb-443d-a6f8-efcaf7e42c76 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:22.296237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:22.397445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff28c42-ecfb-443d-a6f8-efcaf7e42c76 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:22.439856 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e28e6-725f-4d6d-b23b-c0a487daa73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:22.440686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e28e6-725f-4d6d-b23b-c0a487daa73e None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:22.441134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e28e6-725f-4d6d-b23b-c0a487daa73e None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:22.441675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e28e6-725f-4d6d-b23b-c0a487daa73e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:22.462078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e28e6-725f-4d6d-b23b-c0a487daa73e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-fd5e28e6-725f-4d6d-b23b-c0a487daa73e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:45:22.471896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:22.473092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:22.474366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:22.474366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:22.474366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:22.474366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:22.474366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:45:22.474366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:45:22.474366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:45:22.474366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:45:22.474366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:45:22.474366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:45:22.474366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:45:22.474366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:22.474366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 620/1249] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:45:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:22.481661 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef185d0a-9ed0-4bb5-b5b2-894fb8e4b0cf None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:22.604707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef185d0a-9ed0-4bb5-b5b2-894fb8e4b0cf None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:22.641052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:22.647785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537d1d5-3687-44ea-a531-e2a48ec5352c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:22.647785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537d1d5-3687-44ea-a531-e2a48ec5352c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:22.648281 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537d1d5-3687-44ea-a531-e2a48ec5352c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:22.648281 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537d1d5-3687-44ea-a531-e2a48ec5352c None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:22.649013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537d1d5-3687-44ea-a531-e2a48ec5352c None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:22.649319 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537d1d5-3687-44ea-a531-e2a48ec5352c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:22.650703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537d1d5-3687-44ea-a531-e2a48ec5352c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:22.662673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 630/1250] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:45:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:45:22.669776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e28e6-725f-4d6d-b23b-c0a487daa73e None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:22.768503 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:22.795373 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e28e6-725f-4d6d-b23b-c0a487daa73e None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:22.846270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5db40b-dcf3-496c-bf36-ce6647e0ec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:22.847079 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5db40b-dcf3-496c-bf36-ce6647e0ec00 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:22.847405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5db40b-dcf3-496c-bf36-ce6647e0ec00 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:22.847765 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5db40b-dcf3-496c-bf36-ce6647e0ec00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:22.861320 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5db40b-dcf3-496c-bf36-ce6647e0ec00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:22.863435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5db40b-dcf3-496c-bf36-ce6647e0ec00 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-3b5db40b-dcf3-496c-bf36-ce6647e0ec00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:45:22.882561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:22.884712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:22.885741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:45:22.885741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:45:22.885741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:22.885741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:22.885741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 621/1251] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:22.891690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537d1d5-3687-44ea-a531-e2a48ec5352c None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:23.026215 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537d1d5-3687-44ea-a531-e2a48ec5352c None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:23.068384 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8861f-a27e-4396-bbac-c87e6ef311b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:23.069950 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8861f-a27e-4396-bbac-c87e6ef311b0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:23.070237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8861f-a27e-4396-bbac-c87e6ef311b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:23.070515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8861f-a27e-4396-bbac-c87e6ef311b0 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:23.071264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8861f-a27e-4396-bbac-c87e6ef311b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:23.071602 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8861f-a27e-4396-bbac-c87e6ef311b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:23.073024 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8861f-a27e-4396-bbac-c87e6ef311b0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:23.088088 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 631/1252] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:23.094573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5db40b-dcf3-496c-bf36-ce6647e0ec00 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:23.243754 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5db40b-dcf3-496c-bf36-ce6647e0ec00 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:23.275593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:23.284898 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315912cc-1d80-4944-a407-5cdba1debc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:23.285439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315912cc-1d80-4944-a407-5cdba1debc15 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:23.285728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315912cc-1d80-4944-a407-5cdba1debc15 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:23.286007 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315912cc-1d80-4944-a407-5cdba1debc15 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:23.295512 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315912cc-1d80-4944-a407-5cdba1debc15 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:23.297148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315912cc-1d80-4944-a407-5cdba1debc15 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-315912cc-1d80-4944-a407-5cdba1debc15 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:45:23.310458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:23.312071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:23.313240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:23.313240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:23.313240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:45:23.313240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:45:23.313240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:45:23.313240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:45:23.313240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:45:23.313240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:45:23.313240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:45:23.313240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:23.313240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 622/1253] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:23.318360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8861f-a27e-4396-bbac-c87e6ef311b0 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:23.386637 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:23.437713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8861f-a27e-4396-bbac-c87e6ef311b0 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:23.478075 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41be8e-2444-4517-8cb6-ced70754fdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:23.478697 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41be8e-2444-4517-8cb6-ced70754fdf8 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:23.478977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41be8e-2444-4517-8cb6-ced70754fdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:23.479250 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41be8e-2444-4517-8cb6-ced70754fdf8 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:23.480094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41be8e-2444-4517-8cb6-ced70754fdf8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:23.480408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41be8e-2444-4517-8cb6-ced70754fdf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:23.481849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41be8e-2444-4517-8cb6-ced70754fdf8 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:23.500774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 632/1254] 10.222.0.94 () {56 vars in 1143 bytes} [Tue Jul 26 14:45:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:23.507873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315912cc-1d80-4944-a407-5cdba1debc15 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:23.621455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315912cc-1d80-4944-a407-5cdba1debc15 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:23.663782 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f870f4-8068-4bec-be21-8c1d0d163c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:23.676258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f870f4-8068-4bec-be21-8c1d0d163c63 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:23.676258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f870f4-8068-4bec-be21-8c1d0d163c63 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:23.676258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f870f4-8068-4bec-be21-8c1d0d163c63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:23.729355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:23.742673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f870f4-8068-4bec-be21-8c1d0d163c63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:23.744151 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f870f4-8068-4bec-be21-8c1d0d163c63 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:23.858253 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 623/1255] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:45:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:45:23.865037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41be8e-2444-4517-8cb6-ced70754fdf8 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:23.975621 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41be8e-2444-4517-8cb6-ced70754fdf8 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:23.990023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:24.018392 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea217907-0def-499d-8419-32f69d305694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:24.019062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea217907-0def-499d-8419-32f69d305694 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:24.019341 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea217907-0def-499d-8419-32f69d305694 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:24.019647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea217907-0def-499d-8419-32f69d305694 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:24.032756 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea217907-0def-499d-8419-32f69d305694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:24.034213 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea217907-0def-499d-8419-32f69d305694 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-ea217907-0def-499d-8419-32f69d305694 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:45:24.050679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:24.057525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:24.059202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:45:24.059202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:45:24.059202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:24.059202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:24.059202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 633/1256] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:24.059763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f870f4-8068-4bec-be21-8c1d0d163c63 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:24.182908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:24.198278 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f870f4-8068-4bec-be21-8c1d0d163c63 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:24.249012 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74641ec-e759-4885-96f8-54f272d3d5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:24.249804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74641ec-e759-4885-96f8-54f272d3d5df None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:24.250094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74641ec-e759-4885-96f8-54f272d3d5df None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:24.250565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74641ec-e759-4885-96f8-54f272d3d5df None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:24.251559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74641ec-e759-4885-96f8-54f272d3d5df None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:24.251992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74641ec-e759-4885-96f8-54f272d3d5df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:24.253915 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74641ec-e759-4885-96f8-54f272d3d5df None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:24.270530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 624/1257] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:24.280847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea217907-0def-499d-8419-32f69d305694 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:24.421464 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea217907-0def-499d-8419-32f69d305694 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:24.461784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bcda4-2ebd-4a63-939c-554ba6c27490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:24.462459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bcda4-2ebd-4a63-939c-554ba6c27490 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:24.462740 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bcda4-2ebd-4a63-939c-554ba6c27490 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:24.463021 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bcda4-2ebd-4a63-939c-554ba6c27490 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:24.470413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bcda4-2ebd-4a63-939c-554ba6c27490 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:24.475007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bcda4-2ebd-4a63-939c-554ba6c27490 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-f28bcda4-2ebd-4a63-939c-554ba6c27490 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:45:24.484463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:24.485600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:24.486747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:45:24.486747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:45:24.486747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:45:24.486747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:45:24.486747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:45:24.486747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:45:24.486747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:45:24.486747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:24.486747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 634/1258] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:45:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:45:24.494622 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74641ec-e759-4885-96f8-54f272d3d5df None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:24.617299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74641ec-e759-4885-96f8-54f272d3d5df None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:24.659362 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f9f7e-b0c7-43a2-918f-a4fb88bbf6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:24.660062 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f9f7e-b0c7-43a2-918f-a4fb88bbf6e1 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:24.660348 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f9f7e-b0c7-43a2-918f-a4fb88bbf6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:24.660627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f9f7e-b0c7-43a2-918f-a4fb88bbf6e1 None admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:24.661453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f9f7e-b0c7-43a2-918f-a4fb88bbf6e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:24.661753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f9f7e-b0c7-43a2-918f-a4fb88bbf6e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:24.664371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f9f7e-b0c7-43a2-918f-a4fb88bbf6e1 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:24.679521 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 625/1259] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:45:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:24.686771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bcda4-2ebd-4a63-939c-554ba6c27490 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:24.721111 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:24.812776 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bcda4-2ebd-4a63-939c-554ba6c27490 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:24.865535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e8852-678b-482b-915c-71a904b4a678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:24.866601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e8852-678b-482b-915c-71a904b4a678 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:24.866601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e8852-678b-482b-915c-71a904b4a678 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:24.866800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e8852-678b-482b-915c-71a904b4a678 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:24.867597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e8852-678b-482b-915c-71a904b4a678 None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=42a0728e2f6249bdb45d6f44fb15aa4c) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:24.867857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e8852-678b-482b-915c-71a904b4a678 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:24.869367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e8852-678b-482b-915c-71a904b4a678 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:24.875781 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 635/1260] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:45:24 2022] GET /identity/v3/role_assignments?user.id=42a0728e2f6249bdb45d6f44fb15aa4c&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:45:26.987347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f6f23-26ee-49f2-bd8d-d70eb7f088ea None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:26.988025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f6f23-26ee-49f2-bd8d-d70eb7f088ea None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:26.988025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f6f23-26ee-49f2-bd8d-d70eb7f088ea None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:26.989490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 626/1261] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:26.996194 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bb1a3-7252-4644-a796-eb24aa2dd593 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:26.996970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bb1a3-7252-4644-a796-eb24aa2dd593 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:26.996970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bb1a3-7252-4644-a796-eb24aa2dd593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:27.017034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-8e0bb1a3-7252-4644-a796-eb24aa2dd593 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:27.018255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-8e0bb1a3-7252-4644-a796-eb24aa2dd593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:27.104827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 636/1262] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:45:27.128723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f6f23-26ee-49f2-bd8d-d70eb7f088ea None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:27.141218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:27.250397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f6f23-26ee-49f2-bd8d-d70eb7f088ea None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:27.291724 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a4a32-5cf5-46bf-b91d-a07c6c9e93f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:27.292372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a4a32-5cf5-46bf-b91d-a07c6c9e93f8 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:27.292733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a4a32-5cf5-46bf-b91d-a07c6c9e93f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:27.293018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a4a32-5cf5-46bf-b91d-a07c6c9e93f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:27.306064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a4a32-5cf5-46bf-b91d-a07c6c9e93f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-a03a4a32-5cf5-46bf-b91d-a07c6c9e93f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:45:27.311619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:27.312827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:27.313826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:27.313826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:27.313826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:27.313826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:27.313826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:45:27.313826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:45:27.313826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:45:27.313826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:45:27.313826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:45:27.313826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:45:27.313826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:45:27.313826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:27.313826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 627/1263] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:45:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:27.321330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bb1a3-7252-4644-a796-eb24aa2dd593 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:27.342114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:27.442313 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bb1a3-7252-4644-a796-eb24aa2dd593 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:27.483688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3451870-dacb-4396-9242-48a5bebe52b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:27.484386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3451870-dacb-4396-9242-48a5bebe52b2 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:27.484680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3451870-dacb-4396-9242-48a5bebe52b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:27.484949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3451870-dacb-4396-9242-48a5bebe52b2 None admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:27.485677 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3451870-dacb-4396-9242-48a5bebe52b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:27.485986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3451870-dacb-4396-9242-48a5bebe52b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:27.487392 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3451870-dacb-4396-9242-48a5bebe52b2 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:27.497245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 637/1264] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:45:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:27.504337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a4a32-5cf5-46bf-b91d-a07c6c9e93f8 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:27.590561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:27.633878 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a4a32-5cf5-46bf-b91d-a07c6c9e93f8 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:27.673745 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afc875-a266-475d-9f93-f617e62f8340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:27.674636 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afc875-a266-475d-9f93-f617e62f8340 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:27.674923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afc875-a266-475d-9f93-f617e62f8340 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:27.675292 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afc875-a266-475d-9f93-f617e62f8340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:27.687586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afc875-a266-475d-9f93-f617e62f8340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:27.689090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afc875-a266-475d-9f93-f617e62f8340 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-67afc875-a266-475d-9f93-f617e62f8340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:45:27.706258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:27.707428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:27.708384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:45:27.708384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:45:27.708384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:27.708384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:27.708384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 628/1265] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:27.714311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3451870-dacb-4396-9242-48a5bebe52b2 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:27.801672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:27.822334 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3451870-dacb-4396-9242-48a5bebe52b2 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:27.860411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344e748-8ab5-4b37-a6cf-086d4b8195a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:27.861079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344e748-8ab5-4b37-a6cf-086d4b8195a0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:27.861387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344e748-8ab5-4b37-a6cf-086d4b8195a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:27.861700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344e748-8ab5-4b37-a6cf-086d4b8195a0 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:27.862433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344e748-8ab5-4b37-a6cf-086d4b8195a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:27.862834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344e748-8ab5-4b37-a6cf-086d4b8195a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:27.864431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344e748-8ab5-4b37-a6cf-086d4b8195a0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:27.878096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 638/1266] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:27.884943 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afc875-a266-475d-9f93-f617e62f8340 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:27.998045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afc875-a266-475d-9f93-f617e62f8340 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:28.020422 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:28.036213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf7823-1d87-4339-a6de-b6081bc362bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:28.036876 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf7823-1d87-4339-a6de-b6081bc362bb None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:28.037186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf7823-1d87-4339-a6de-b6081bc362bb None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:28.037460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf7823-1d87-4339-a6de-b6081bc362bb None admin] PATH_INFO: `/v3/users/placement` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:28.047035 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf7823-1d87-4339-a6de-b6081bc362bb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:28.048740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf7823-1d87-4339-a6de-b6081bc362bb None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-a9cf7823-1d87-4339-a6de-b6081bc362bb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:45:28.062378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:28.063684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:28.065254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:28.065254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:28.065254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:45:28.065254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:45:28.065254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:45:28.065254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:45:28.065254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:45:28.065254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:45:28.065254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:45:28.065254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:28.065254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 629/1267] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:28.070301 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344e748-8ab5-4b37-a6cf-086d4b8195a0 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:28.198671 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344e748-8ab5-4b37-a6cf-086d4b8195a0 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:28.235809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ae17f-52f2-4532-81f8-9614cbbf20e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:28.236491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ae17f-52f2-4532-81f8-9614cbbf20e4 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:28.236811 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ae17f-52f2-4532-81f8-9614cbbf20e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:28.237128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ae17f-52f2-4532-81f8-9614cbbf20e4 None admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:28.237982 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ae17f-52f2-4532-81f8-9614cbbf20e4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:28.238332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ae17f-52f2-4532-81f8-9614cbbf20e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:28.239836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ae17f-52f2-4532-81f8-9614cbbf20e4 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:28.251906 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:28.267755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 639/1268] 10.222.0.94 () {56 vars in 1143 bytes} [Tue Jul 26 14:45:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:28.273650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf7823-1d87-4339-a6de-b6081bc362bb None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:28.388190 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf7823-1d87-4339-a6de-b6081bc362bb None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:28.427118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e5e69-58ef-4431-866e-c13b0fa24884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:28.437465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e5e69-58ef-4431-866e-c13b0fa24884 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:28.437465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e5e69-58ef-4431-866e-c13b0fa24884 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:28.437465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e5e69-58ef-4431-866e-c13b0fa24884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:28.440365 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e5e69-58ef-4431-866e-c13b0fa24884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:28.441851 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e5e69-58ef-4431-866e-c13b0fa24884 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-250e5e69-58ef-4431-866e-c13b0fa24884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:45:28.458567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:28.460496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:28.461692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:45:28.461692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:45:28.461692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:28.461692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:28.461692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 630/1269] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:28.466788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ae17f-52f2-4532-81f8-9614cbbf20e4 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:28.606061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ae17f-52f2-4532-81f8-9614cbbf20e4 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:28.641686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0dde4b-505f-45d5-bfdc-121736447f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:28.642224 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0dde4b-505f-45d5-bfdc-121736447f49 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:28.642498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0dde4b-505f-45d5-bfdc-121736447f49 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:28.642766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0dde4b-505f-45d5-bfdc-121736447f49 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:28.643443 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0dde4b-505f-45d5-bfdc-121736447f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:28.643771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0dde4b-505f-45d5-bfdc-121736447f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:28.645127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0dde4b-505f-45d5-bfdc-121736447f49 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:28.657447 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 640/1270] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:45:28.664636 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e5e69-58ef-4431-866e-c13b0fa24884 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:28.702373 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:28.791685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e5e69-58ef-4431-866e-c13b0fa24884 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:28.831928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c02ea-e05d-4a55-94d2-bbcb9592a09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:28.835384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c02ea-e05d-4a55-94d2-bbcb9592a09b None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:28.835384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c02ea-e05d-4a55-94d2-bbcb9592a09b None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:28.835384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c02ea-e05d-4a55-94d2-bbcb9592a09b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:28.848099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c02ea-e05d-4a55-94d2-bbcb9592a09b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:28.850824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c02ea-e05d-4a55-94d2-bbcb9592a09b None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-6a4c02ea-e05d-4a55-94d2-bbcb9592a09b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:45:28.873268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:28.874437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:28.883940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:45:28.883940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:45:28.883940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:45:28.883940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:45:28.883940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:45:28.883940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:45:28.883940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:45:28.883940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:28.883940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 631/1271] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:45:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:28.884578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0dde4b-505f-45d5-bfdc-121736447f49 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:28.951618 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:29.002181 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0dde4b-505f-45d5-bfdc-121736447f49 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:29.042873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2f087-5e39-49ec-84da-a4215b19e9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:29.043673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f087-5e39-49ec-84da-a4215b19e9f9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:29.044037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f087-5e39-49ec-84da-a4215b19e9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:29.044396 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f087-5e39-49ec-84da-a4215b19e9f9 None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:29.045270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2f087-5e39-49ec-84da-a4215b19e9f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:29.045665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2f087-5e39-49ec-84da-a4215b19e9f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:29.047262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2f087-5e39-49ec-84da-a4215b19e9f9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:29.060419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 641/1272] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:45:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:29.068929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c02ea-e05d-4a55-94d2-bbcb9592a09b None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:29.170241 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:29.180948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c02ea-e05d-4a55-94d2-bbcb9592a09b None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:29.217947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c0604-0693-493a-b40e-f59f4df6b2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:29.218631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c0604-0693-493a-b40e-f59f4df6b2f4 None admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:29.218976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c0604-0693-493a-b40e-f59f4df6b2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:29.219302 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c0604-0693-493a-b40e-f59f4df6b2f4 None admin] PATH_INFO: `/v3/projects/a9b328041da24cedbc801b271d6e0419/users/42a0728e2f6249bdb45d6f44fb15aa4c/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:29.221843 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c0604-0693-493a-b40e-f59f4df6b2f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9b328041da24cedbc801b271d6e0419, user_id=42a0728e2f6249bdb45d6f44fb15aa4c, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:29.223519 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c0604-0693-493a-b40e-f59f4df6b2f4 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:29.337237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 632/1273] 10.222.0.94 () {58 vars in 1439 bytes} [Tue Jul 26 14:45:29 2022] PUT /identity/v3/projects/a9b328041da24cedbc801b271d6e0419/users/42a0728e2f6249bdb45d6f44fb15aa4c/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 14:45:31.570038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba03f12-12c5-49c0-a684-576d798a25be None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:31.570835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba03f12-12c5-49c0-a684-576d798a25be None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:31.570936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba03f12-12c5-49c0-a684-576d798a25be None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:31.573515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 642/1274] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:45:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:31.580210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f387fb-881d-4867-983f-6854e98f5811 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:31.580729 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f387fb-881d-4867-983f-6854e98f5811 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:31.581125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f387fb-881d-4867-983f-6854e98f5811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:31.603508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-57f387fb-881d-4867-983f-6854e98f5811 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:31.604902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-57f387fb-881d-4867-983f-6854e98f5811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:31.689535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 633/1275] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:45:31.717046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba03f12-12c5-49c0-a684-576d798a25be None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:31.895926 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba03f12-12c5-49c0-a684-576d798a25be None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:31.936299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:31.970383 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd02904-028f-434c-a835-e7e8b93e9a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:31.971566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd02904-028f-434c-a835-e7e8b93e9a15 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:31.972027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd02904-028f-434c-a835-e7e8b93e9a15 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:31.972464 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd02904-028f-434c-a835-e7e8b93e9a15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:31.994583 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd02904-028f-434c-a835-e7e8b93e9a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-ebd02904-028f-434c-a835-e7e8b93e9a15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 14:45:32.005410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:32.009164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:32.010745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:32.010745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:32.010745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:32.010745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:32.010745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 14:45:32.010745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 14:45:32.010745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 14:45:32.010745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 14:45:32.010745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 14:45:32.010745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 14:45:32.010745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 14:45:32.010745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:32.010745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 643/1276] 10.222.0.94 () {56 vars in 1096 bytes} [Tue Jul 26 14:45:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:32.018574 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f387fb-881d-4867-983f-6854e98f5811 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:32.150723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:32.182013 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f387fb-881d-4867-983f-6854e98f5811 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:32.247763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28d3b7-a86b-4503-8914-7226d11ff4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:32.248831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28d3b7-a86b-4503-8914-7226d11ff4a5 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:32.249311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28d3b7-a86b-4503-8914-7226d11ff4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:32.249791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28d3b7-a86b-4503-8914-7226d11ff4a5 None admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:32.251111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28d3b7-a86b-4503-8914-7226d11ff4a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:32.251636 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28d3b7-a86b-4503-8914-7226d11ff4a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:32.259875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28d3b7-a86b-4503-8914-7226d11ff4a5 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:32.270216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 634/1277] 10.222.0.94 () {56 vars in 1099 bytes} [Tue Jul 26 14:45:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:32.277383 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd02904-028f-434c-a835-e7e8b93e9a15 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:32.411278 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd02904-028f-434c-a835-e7e8b93e9a15 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:32.452849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a9941-f8c0-41ee-863b-1bdd576e44db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:32.453495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a9941-f8c0-41ee-863b-1bdd576e44db None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:32.453776 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a9941-f8c0-41ee-863b-1bdd576e44db None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:32.454035 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a9941-f8c0-41ee-863b-1bdd576e44db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:32.466582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a9941-f8c0-41ee-863b-1bdd576e44db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:32.469656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a9941-f8c0-41ee-863b-1bdd576e44db None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-489a9941-f8c0-41ee-863b-1bdd576e44db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:45:32.484572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:32.486398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:32.488243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:45:32.488243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:45:32.488243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:32.488243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:32.488243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 644/1278] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:32.493388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28d3b7-a86b-4503-8914-7226d11ff4a5 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:32.625299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28d3b7-a86b-4503-8914-7226d11ff4a5 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:32.669465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa430b8-32e9-4cfe-b9d8-3e8c82a39939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:32.670082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa430b8-32e9-4cfe-b9d8-3e8c82a39939 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:32.670412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa430b8-32e9-4cfe-b9d8-3e8c82a39939 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:32.670726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa430b8-32e9-4cfe-b9d8-3e8c82a39939 None admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:32.671530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa430b8-32e9-4cfe-b9d8-3e8c82a39939 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:32.671916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa430b8-32e9-4cfe-b9d8-3e8c82a39939 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:32.673429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa430b8-32e9-4cfe-b9d8-3e8c82a39939 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:32.689752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 635/1279] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:32.696790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a9941-f8c0-41ee-863b-1bdd576e44db None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:32.741983 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:32.876425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a9941-f8c0-41ee-863b-1bdd576e44db None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:32.941378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4c4ee-d453-4fc1-8320-683565ab4592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:32.942360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4c4ee-d453-4fc1-8320-683565ab4592 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:32.942852 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4c4ee-d453-4fc1-8320-683565ab4592 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:32.944074 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4c4ee-d453-4fc1-8320-683565ab4592 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:32.960243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4c4ee-d453-4fc1-8320-683565ab4592 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:32.962940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4c4ee-d453-4fc1-8320-683565ab4592 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-fae4c4ee-d453-4fc1-8320-683565ab4592 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 14:45:32.986106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:32.987423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:32.988658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:32.988658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:32.988658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 14:45:32.988658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 14:45:32.988658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 14:45:32.988658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 14:45:32.988658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 14:45:32.988658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 14:45:32.988658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 14:45:32.988658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:32.988658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 645/1280] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:32.995607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa430b8-32e9-4cfe-b9d8-3e8c82a39939 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:33.000332 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:33.130009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa430b8-32e9-4cfe-b9d8-3e8c82a39939 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:33.170532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04e05e-60a7-41db-a943-fa5515253159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:33.172530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04e05e-60a7-41db-a943-fa5515253159 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:33.172530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04e05e-60a7-41db-a943-fa5515253159 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:33.172530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04e05e-60a7-41db-a943-fa5515253159 None admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:33.172813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04e05e-60a7-41db-a943-fa5515253159 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:33.172915 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04e05e-60a7-41db-a943-fa5515253159 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:33.174463 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04e05e-60a7-41db-a943-fa5515253159 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:33.195998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 636/1281] 10.222.0.94 () {56 vars in 1143 bytes} [Tue Jul 26 14:45:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 14:45:33.205257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4c4ee-d453-4fc1-8320-683565ab4592 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:33.328213 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:33.340450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4c4ee-d453-4fc1-8320-683565ab4592 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:33.378787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc587f31-dd7c-4715-b1f9-cda1a81591e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:33.379520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc587f31-dd7c-4715-b1f9-cda1a81591e1 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:33.379877 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc587f31-dd7c-4715-b1f9-cda1a81591e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:33.380183 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc587f31-dd7c-4715-b1f9-cda1a81591e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:33.463228 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc587f31-dd7c-4715-b1f9-cda1a81591e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:33.465161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc587f31-dd7c-4715-b1f9-cda1a81591e1 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:33.598680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:33.619213 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 646/1282] 10.222.0.94 () {58 vars in 1282 bytes} [Tue Jul 26 14:45:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 14:45:33.625376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04e05e-60a7-41db-a943-fa5515253159 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:33.704465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:33.752419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04e05e-60a7-41db-a943-fa5515253159 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:33.795890 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b3c33-d9f7-45ef-b9b2-26aeb15d09e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:33.796713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b3c33-d9f7-45ef-b9b2-26aeb15d09e0 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:33.797086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b3c33-d9f7-45ef-b9b2-26aeb15d09e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:33.797493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b3c33-d9f7-45ef-b9b2-26aeb15d09e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:33.810540 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b3c33-d9f7-45ef-b9b2-26aeb15d09e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:33.812542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b3c33-d9f7-45ef-b9b2-26aeb15d09e0 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-3a1b3c33-d9f7-45ef-b9b2-26aeb15d09e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 14:45:33.830000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:33.836696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:33.838637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 14:45:33.838637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 14:45:33.838637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 14:45:33.838637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:33.838637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 637/1283] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:45:33.839370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc587f31-dd7c-4715-b1f9-cda1a81591e1 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:33.961655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc587f31-dd7c-4715-b1f9-cda1a81591e1 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:34.011000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ff05c-55f6-429a-ad0a-a348f6957ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:34.011586 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ff05c-55f6-429a-ad0a-a348f6957ce9 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:34.011876 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ff05c-55f6-429a-ad0a-a348f6957ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:34.012149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ff05c-55f6-429a-ad0a-a348f6957ce9 None admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:34.013204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ff05c-55f6-429a-ad0a-a348f6957ce9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:34.013784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ff05c-55f6-429a-ad0a-a348f6957ce9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:34.016509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ff05c-55f6-429a-ad0a-a348f6957ce9 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:34.041610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 647/1284] 10.222.0.94 () {56 vars in 1109 bytes} [Tue Jul 26 14:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:34.050378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b3c33-d9f7-45ef-b9b2-26aeb15d09e0 None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:34.247082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b3c33-d9f7-45ef-b9b2-26aeb15d09e0 None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:34.267275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:34.299747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a559f78-a95d-4c50-975b-1d7cd3796a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:34.300871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a559f78-a95d-4c50-975b-1d7cd3796a3d None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:34.301381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a559f78-a95d-4c50-975b-1d7cd3796a3d None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:34.301877 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a559f78-a95d-4c50-975b-1d7cd3796a3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:34.315354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a559f78-a95d-4c50-975b-1d7cd3796a3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:34.318705 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a559f78-a95d-4c50-975b-1d7cd3796a3d None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application [None req-2a559f78-a95d-4c50-975b-1d7cd3796a3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 14:45:34.342381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:34.343619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:34.344766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 14:45:34.344766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 14:45:34.344766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 14:45:34.344766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 14:45:34.344766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 14:45:34.344766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 14:45:34.344766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 14:45:34.344766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: ERROR keystone.server.flask.application Jul 26 14:45:34.345453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 638/1285] 10.222.0.94 () {56 vars in 1111 bytes} [Tue Jul 26 14:45:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:34.353038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ff05c-55f6-429a-ad0a-a348f6957ce9 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:34.413717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:34.473748 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ff05c-55f6-429a-ad0a-a348f6957ce9 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:34.514402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d287666-6335-408f-ad88-f46b6249f4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:34.515085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d287666-6335-408f-ad88-f46b6249f4a0 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:34.515479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d287666-6335-408f-ad88-f46b6249f4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:34.515826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d287666-6335-408f-ad88-f46b6249f4a0 None admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:34.516734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d287666-6335-408f-ad88-f46b6249f4a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:34.517098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d287666-6335-408f-ad88-f46b6249f4a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:34.518739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d287666-6335-408f-ad88-f46b6249f4a0 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:34.533437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 648/1286] 10.222.0.94 () {56 vars in 1148 bytes} [Tue Jul 26 14:45:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:34.541089 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a559f78-a95d-4c50-975b-1d7cd3796a3d None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:34.662649 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a559f78-a95d-4c50-975b-1d7cd3796a3d None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:34.711920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3dab0-3115-40c5-a30c-3bd4216cf5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:34.712563 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3dab0-3115-40c5-a30c-3bd4216cf5f9 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:34.712855 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3dab0-3115-40c5-a30c-3bd4216cf5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:34.713148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3dab0-3115-40c5-a30c-3bd4216cf5f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:34.713960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3dab0-3115-40c5-a30c-3bd4216cf5f9 None admin] RBAC: Adding query filter params (role.id=2bb293e71ec0464fb7f6fa454e6b3f51, scope.project.id=a9b328041da24cedbc801b271d6e0419, user.id=42a0728e2f6249bdb45d6f44fb15aa4c) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:34.714276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3dab0-3115-40c5-a30c-3bd4216cf5f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:34.715910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3dab0-3115-40c5-a30c-3bd4216cf5f9 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:34.723336 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 639/1287] 10.222.0.94 () {56 vars in 1374 bytes} [Tue Jul 26 14:45:34 2022] GET /identity/v3/role_assignments?user.id=42a0728e2f6249bdb45d6f44fb15aa4c&scope.project.id=a9b328041da24cedbc801b271d6e0419&role.id=2bb293e71ec0464fb7f6fa454e6b3f51 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:36.951432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24327ab1-e09e-40c9-863d-98ea53e1e21c None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:36.952094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24327ab1-e09e-40c9-863d-98ea53e1e21c None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:36.952345 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24327ab1-e09e-40c9-863d-98ea53e1e21c None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:36.954357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 649/1288] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:45:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:36.962725 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af99dc2-f4c0-4457-9a7e-5158ff946de4 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:36.963263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af99dc2-f4c0-4457-9a7e-5158ff946de4 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:36.963750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af99dc2-f4c0-4457-9a7e-5158ff946de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:36.986355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-0af99dc2-f4c0-4457-9a7e-5158ff946de4 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:36.987636 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0af99dc2-f4c0-4457-9a7e-5158ff946de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:37.028984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:37.079907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 640/1289] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:45:37.103565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24327ab1-e09e-40c9-863d-98ea53e1e21c None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:37.230355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24327ab1-e09e-40c9-863d-98ea53e1e21c None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:37.242563 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:37.270436 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7e54a-50bf-463a-85d0-277910c21061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:37.271725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7e54a-50bf-463a-85d0-277910c21061 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:37.272002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7e54a-50bf-463a-85d0-277910c21061 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:37.272273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7e54a-50bf-463a-85d0-277910c21061 None admin] PATH_INFO: `/v3/services/placement` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:37.273000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7e54a-50bf-463a-85d0-277910c21061 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:37.274495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7e54a-50bf-463a-85d0-277910c21061 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-d2a7e54a-50bf-463a-85d0-277910c21061 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:37.283972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:37.286160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:37.288489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 650/1290] 10.222.0.94 () {56 vars in 1117 bytes} [Tue Jul 26 14:45:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 14:45:37.294765 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af99dc2-f4c0-4457-9a7e-5158ff946de4 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:37.415801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af99dc2-f4c0-4457-9a7e-5158ff946de4 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:37.458125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4ff74-2887-46ad-ae42-7393b55cdf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:37.459105 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ff74-2887-46ad-ae42-7393b55cdf20 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:37.459400 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ff74-2887-46ad-ae42-7393b55cdf20 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:37.459747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ff74-2887-46ad-ae42-7393b55cdf20 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:37.460499 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4ff74-2887-46ad-ae42-7393b55cdf20 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:37.460875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4ff74-2887-46ad-ae42-7393b55cdf20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:37.462385 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4ff74-2887-46ad-ae42-7393b55cdf20 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:37.468328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 641/1291] 10.222.0.94 () {56 vars in 1116 bytes} [Tue Jul 26 14:45:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:37.477574 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7e54a-50bf-463a-85d0-277910c21061 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:37.608522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7e54a-50bf-463a-85d0-277910c21061 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:37.654481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ea5ab-52ec-47c6-aaeb-eba97a26dc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:37.655597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ea5ab-52ec-47c6-aaeb-eba97a26dc8c None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:37.656062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ea5ab-52ec-47c6-aaeb-eba97a26dc8c None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:37.656485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ea5ab-52ec-47c6-aaeb-eba97a26dc8c None admin] PATH_INFO: `/v3/services` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:37.657674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ea5ab-52ec-47c6-aaeb-eba97a26dc8c None admin] RBAC: Adding query filter params (type=placement) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:37.658198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ea5ab-52ec-47c6-aaeb-eba97a26dc8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:37.660674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ea5ab-52ec-47c6-aaeb-eba97a26dc8c None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:37.669336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 651/1292] 10.222.0.94 () {56 vars in 1116 bytes} [Tue Jul 26 14:45:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:40.112167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf027117-9efe-43aa-b06b-9934a58aa965 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:40.113023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf027117-9efe-43aa-b06b-9934a58aa965 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:40.113023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf027117-9efe-43aa-b06b-9934a58aa965 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:40.114433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 642/1293] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:45:40.121488 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbb1ec-0eae-4a0a-ae9c-4bcf7fb43410 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:40.122092 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbb1ec-0eae-4a0a-ae9c-4bcf7fb43410 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:40.122092 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbb1ec-0eae-4a0a-ae9c-4bcf7fb43410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:40.142640 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7edbb1ec-0eae-4a0a-ae9c-4bcf7fb43410 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:40.143890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7edbb1ec-0eae-4a0a-ae9c-4bcf7fb43410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:40.179275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:40.222237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 652/1294] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:45:40.264388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf027117-9efe-43aa-b06b-9934a58aa965 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:40.300025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:40.386894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf027117-9efe-43aa-b06b-9934a58aa965 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:40.428034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62eca97-c79e-4a07-a61e-8d9cddd2274c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:40.428681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62eca97-c79e-4a07-a61e-8d9cddd2274c None admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:40.429563 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62eca97-c79e-4a07-a61e-8d9cddd2274c None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:40.429563 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62eca97-c79e-4a07-a61e-8d9cddd2274c None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:40.430121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62eca97-c79e-4a07-a61e-8d9cddd2274c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:40.432485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62eca97-c79e-4a07-a61e-8d9cddd2274c None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:40.540716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 643/1295] 10.222.0.94 () {60 vars in 1141 bytes} [Tue Jul 26 14:45:40 2022] POST /identity/v3/services => generated 246 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:45:42.822179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228add9-7049-4d9c-a0da-b1feb21c8fc7 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:42.822843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228add9-7049-4d9c-a0da-b1feb21c8fc7 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:42.822843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228add9-7049-4d9c-a0da-b1feb21c8fc7 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:42.824446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 653/1296] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:45:42.831008 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcca1fe-9982-455e-9f53-2cf1418104e4 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:42.831684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcca1fe-9982-455e-9f53-2cf1418104e4 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:42.831839 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcca1fe-9982-455e-9f53-2cf1418104e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:42.854727 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-efcca1fe-9982-455e-9f53-2cf1418104e4 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:42.856276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-efcca1fe-9982-455e-9f53-2cf1418104e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:42.954689 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 644/1297] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:42 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 14:45:42.981115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228add9-7049-4d9c-a0da-b1feb21c8fc7 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:43.106868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228add9-7049-4d9c-a0da-b1feb21c8fc7 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:43.156417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33315389-cd52-4e16-829f-79a26ced3b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:43.157061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33315389-cd52-4e16-829f-79a26ced3b51 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:43.157339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33315389-cd52-4e16-829f-79a26ced3b51 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:43.158076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33315389-cd52-4e16-829f-79a26ced3b51 None admin] PATH_INFO: `/v3/services/placement` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:43.158248 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33315389-cd52-4e16-829f-79a26ced3b51 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:43.159641 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33315389-cd52-4e16-829f-79a26ced3b51 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-33315389-cd52-4e16-829f-79a26ced3b51 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:43.169610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:43.171868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:43.173965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 654/1298] 10.222.0.94 () {56 vars in 1117 bytes} [Tue Jul 26 14:45:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:43.178626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcca1fe-9982-455e-9f53-2cf1418104e4 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:43.203761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:43.305468 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcca1fe-9982-455e-9f53-2cf1418104e4 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:43.355182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfc5a9-cf5c-44b4-8fce-24470bf89a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:43.356149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfc5a9-cf5c-44b4-8fce-24470bf89a1d None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:43.356584 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfc5a9-cf5c-44b4-8fce-24470bf89a1d None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:43.357057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfc5a9-cf5c-44b4-8fce-24470bf89a1d None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:43.358108 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfc5a9-cf5c-44b4-8fce-24470bf89a1d None admin] RBAC: Adding query filter params (name=placement) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:43.358584 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfc5a9-cf5c-44b4-8fce-24470bf89a1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:43.360946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfc5a9-cf5c-44b4-8fce-24470bf89a1d None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:43.370782 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 645/1299] 10.222.0.94 () {56 vars in 1116 bytes} [Tue Jul 26 14:45:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:43.378629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33315389-cd52-4e16-829f-79a26ced3b51 None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:43.438116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:43.529636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33315389-cd52-4e16-829f-79a26ced3b51 None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:43.571626 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d18830-e2b4-42a4-9f8f-af35b0837ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:43.573037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d18830-e2b4-42a4-9f8f-af35b0837ce1 None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:43.573520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d18830-e2b4-42a4-9f8f-af35b0837ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:43.574049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d18830-e2b4-42a4-9f8f-af35b0837ce1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:43.575590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d18830-e2b4-42a4-9f8f-af35b0837ce1 None admin] RBAC: Adding query filter params (interface=public, service_id=3bedf6e161ea48b7a1ac8c01341d5d25, region_id=RegionOne) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:43.576091 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d18830-e2b4-42a4-9f8f-af35b0837ce1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:43.578178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d18830-e2b4-42a4-9f8f-af35b0837ce1 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:43.585706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 655/1300] 10.222.0.94 () {56 vars in 1251 bytes} [Tue Jul 26 14:45:43 2022] GET /identity/v3/endpoints?service_id=3bedf6e161ea48b7a1ac8c01341d5d25&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:43.594156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfc5a9-cf5c-44b4-8fce-24470bf89a1d None admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:43.698062 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:43.720129 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfc5a9-cf5c-44b4-8fce-24470bf89a1d None admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:43.760245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2c5a0-f9f0-4595-95ad-84ce2d81a617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:43.760884 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2c5a0-f9f0-4595-95ad-84ce2d81a617 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:43.761173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2c5a0-f9f0-4595-95ad-84ce2d81a617 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:43.761439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2c5a0-f9f0-4595-95ad-84ce2d81a617 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:43.762122 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2c5a0-f9f0-4595-95ad-84ce2d81a617 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:43.763580 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2c5a0-f9f0-4595-95ad-84ce2d81a617 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:43.772491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 646/1301] 10.222.0.94 () {56 vars in 1087 bytes} [Tue Jul 26 14:45:43 2022] GET /identity/v3/services => generated 2191 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:45:46.015271 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ddb44-92d9-4d7d-a762-39e26cb858f1 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:46.015899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ddb44-92d9-4d7d-a762-39e26cb858f1 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:46.015899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ddb44-92d9-4d7d-a762-39e26cb858f1 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:46.017214 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 656/1302] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:45:46.024768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcc782-0455-49c6-8925-0d01e6fbd9db None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:46.025383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcc782-0455-49c6-8925-0d01e6fbd9db None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:46.025383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcc782-0455-49c6-8925-0d01e6fbd9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:46.046259 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-02dcc782-0455-49c6-8925-0d01e6fbd9db None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:45:46.047515 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-02dcc782-0455-49c6-8925-0d01e6fbd9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:45:46.127753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 647/1303] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:45:46 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 14:45:46.151649 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ddb44-92d9-4d7d-a762-39e26cb858f1 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:46.274809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:46.283138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ddb44-92d9-4d7d-a762-39e26cb858f1 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:46.324896 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661182b-3b4d-42fe-a60c-0be4f24041bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:46.325716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661182b-3b4d-42fe-a60c-0be4f24041bd None admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:46.326071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661182b-3b4d-42fe-a60c-0be4f24041bd None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:46.326436 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661182b-3b4d-42fe-a60c-0be4f24041bd None admin] PATH_INFO: `/v3/services/placement` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:46.327159 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661182b-3b4d-42fe-a60c-0be4f24041bd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:46.328786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661182b-3b4d-42fe-a60c-0be4f24041bd None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application [None req-b661182b-3b4d-42fe-a60c-0be4f24041bd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 14:45:46.340288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 14:45:46.347129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application with Lock( Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self._enter() Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application return self.creator() Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application created_value = creator( Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: ERROR keystone.server.flask.application Jul 26 14:45:46.349805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 657/1304] 10.222.0.94 () {56 vars in 1117 bytes} [Tue Jul 26 14:45:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 14:45:46.351857 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcc782-0455-49c6-8925-0d01e6fbd9db None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:46.456126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:46.487220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcc782-0455-49c6-8925-0d01e6fbd9db None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:46.530236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e333a0d-c263-4830-9b0c-750fb5b86289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:46.530793 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e333a0d-c263-4830-9b0c-750fb5b86289 None admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:46.531084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e333a0d-c263-4830-9b0c-750fb5b86289 None admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:46.531634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e333a0d-c263-4830-9b0c-750fb5b86289 None admin] PATH_INFO: `/v3/services` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:46.532263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e333a0d-c263-4830-9b0c-750fb5b86289 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 14:45:46.532659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e333a0d-c263-4830-9b0c-750fb5b86289 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:46.534145 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e333a0d-c263-4830-9b0c-750fb5b86289 None admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:46.541370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 648/1305] 10.222.0.94 () {56 vars in 1116 bytes} [Tue Jul 26 14:45:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 14:45:46.548088 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661182b-3b4d-42fe-a60c-0be4f24041bd None admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:45:46.710157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661182b-3b4d-42fe-a60c-0be4f24041bd None admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:45:46.752908 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ff879-3a65-40bf-bb51-b91e51b77d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:45:46.753700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ff879-3a65-40bf-bb51-b91e51b77d31 None admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:45:46.753992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ff879-3a65-40bf-bb51-b91e51b77d31 None admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:45:46.754312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ff879-3a65-40bf-bb51-b91e51b77d31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:45:46.755684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ff879-3a65-40bf-bb51-b91e51b77d31 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:45:46.757605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ff879-3a65-40bf-bb51-b91e51b77d31 None admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:45:46.916681 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 658/1306] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 14:45:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 14:45:56.589050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=30 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:45:56.908328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:44.518416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c4fca-7d54-4449-84fd-f0d951bb7266 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:44.518987 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c4fca-7d54-4449-84fd-f0d951bb7266 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:44.519297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c4fca-7d54-4449-84fd-f0d951bb7266 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:44.521788 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 649/1307] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:54:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:54:44.529361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925032-e3a8-4561-9f75-6edfaebff126 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:44.529937 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925032-e3a8-4561-9f75-6edfaebff126 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:44.530274 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925032-e3a8-4561-9f75-6edfaebff126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:44.717127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-75925032-e3a8-4561-9f75-6edfaebff126 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:44.718863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-75925032-e3a8-4561-9f75-6edfaebff126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:45.023710 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 659/1308] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:45.121962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70c2cd-2a10-4aec-94bf-c45150f1cfe2 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:45.122670 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70c2cd-2a10-4aec-94bf-c45150f1cfe2 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:45.123182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70c2cd-2a10-4aec-94bf-c45150f1cfe2 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:45.124977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 650/1309] 10.222.0.94 () {54 vars in 965 bytes} [Tue Jul 26 14:54:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:54:45.135858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef9c63-5c5d-4068-a75e-082a16bb3148 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:45.136211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef9c63-5c5d-4068-a75e-082a16bb3148 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:45.136431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef9c63-5c5d-4068-a75e-082a16bb3148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:45.240070 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-80ef9c63-5c5d-4068-a75e-082a16bb3148 None None] MFA Rules not processed for user `d39fd2708591424ca4892973e7da48ee`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:45.242446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-80ef9c63-5c5d-4068-a75e-082a16bb3148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:45.284479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:45.367966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 660/1310] 10.222.0.94 () {58 vars in 1062 bytes} [Tue Jul 26 14:54:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:45.379881 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70c2cd-2a10-4aec-94bf-c45150f1cfe2 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:45.381687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-2f70c2cd-2a10-4aec-94bf-c45150f1cfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:45.533863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70c2cd-2a10-4aec-94bf-c45150f1cfe2 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:45.646073 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ea718-a58b-4ba1-9a30-f95f628fc04b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:45.647034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ea718-a58b-4ba1-9a30-f95f628fc04b service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:45.647496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ea718-a58b-4ba1-9a30-f95f628fc04b service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:45.647934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ea718-a58b-4ba1-9a30-f95f628fc04b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:45.650057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e86ea718-a58b-4ba1-9a30-f95f628fc04b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:45.728788 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:45.815805 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ea718-a58b-4ba1-9a30-f95f628fc04b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:45.818016 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ea718-a58b-4ba1-9a30-f95f628fc04b service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:45.855972 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 651/1311] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:48.504103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3953a2d-2be8-4ca2-bb25-7b0ddc0622fe None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:48.504963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3953a2d-2be8-4ca2-bb25-7b0ddc0622fe None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:48.504963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3953a2d-2be8-4ca2-bb25-7b0ddc0622fe None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:48.506101 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 661/1312] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:54:48.511812 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e05e0-c738-4af8-8daa-46b601e7dc64 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:48.512413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e05e0-c738-4af8-8daa-46b601e7dc64 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:48.512413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e05e0-c738-4af8-8daa-46b601e7dc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:48.610598 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-0d1e05e0-c738-4af8-8daa-46b601e7dc64 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:48.612556 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0d1e05e0-c738-4af8-8daa-46b601e7dc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:48.723723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 652/1313] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:48.765416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3953a2d-2be8-4ca2-bb25-7b0ddc0622fe None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:48.767804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f3953a2d-2be8-4ca2-bb25-7b0ddc0622fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:48.883071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3953a2d-2be8-4ca2-bb25-7b0ddc0622fe None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:48.991067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:48.993565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9dfd8b-ee79-48ec-9ee2-f9c1bcab5a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:48.994184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dfd8b-ee79-48ec-9ee2-f9c1bcab5a13 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:48.994459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dfd8b-ee79-48ec-9ee2-f9c1bcab5a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:48.994731 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dfd8b-ee79-48ec-9ee2-f9c1bcab5a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:48.996412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-cf9dfd8b-ee79-48ec-9ee2-f9c1bcab5a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:49.119532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9dfd8b-ee79-48ec-9ee2-f9c1bcab5a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:49.120989 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9dfd8b-ee79-48ec-9ee2-f9c1bcab5a13 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:49.144566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 662/1314] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:54:51.979034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e30bc6-0215-4004-8895-2ae2e22dac17 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:51.979683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e30bc6-0215-4004-8895-2ae2e22dac17 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:51.979780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e30bc6-0215-4004-8895-2ae2e22dac17 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:51.982345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 653/1315] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:54:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:54:51.989984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813f19d-53ac-4a7b-bbf5-b2f62885a3cf None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:51.990548 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813f19d-53ac-4a7b-bbf5-b2f62885a3cf None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:51.990881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813f19d-53ac-4a7b-bbf5-b2f62885a3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:52.080475 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-6813f19d-53ac-4a7b-bbf5-b2f62885a3cf None None] MFA Rules not processed for user `843d8bc8740f48aaa93c0f79dc9687d0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:52.081751 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6813f19d-53ac-4a7b-bbf5-b2f62885a3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:52.142869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:52.199451 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 663/1316] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:54:51 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:52.260269 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b48a1-af46-40c6-978e-c0e6774d1c3b None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:52.260831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b48a1-af46-40c6-978e-c0e6774d1c3b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:52.261346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b48a1-af46-40c6-978e-c0e6774d1c3b None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:52.264317 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 654/1317] 10.222.0.94 () {54 vars in 965 bytes} [Tue Jul 26 14:54:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:54:52.273674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeca6e3-a2e9-40bd-a263-90c89c6493c2 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:52.273954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeca6e3-a2e9-40bd-a263-90c89c6493c2 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:52.274225 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeca6e3-a2e9-40bd-a263-90c89c6493c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:52.367933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-8eeca6e3-a2e9-40bd-a263-90c89c6493c2 None None] MFA Rules not processed for user `d39fd2708591424ca4892973e7da48ee`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:52.369178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-8eeca6e3-a2e9-40bd-a263-90c89c6493c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:52.471904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 664/1318] 10.222.0.94 () {58 vars in 1062 bytes} [Tue Jul 26 14:54:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:54:52.481060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b48a1-af46-40c6-978e-c0e6774d1c3b None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:52.482614 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-484b48a1-af46-40c6-978e-c0e6774d1c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:52.492838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:52.640842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b48a1-af46-40c6-978e-c0e6774d1c3b None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:52.732345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe71800-1661-4dc4-8fad-0a0b4eb06e75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:52.732967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe71800-1661-4dc4-8fad-0a0b4eb06e75 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:52.733247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe71800-1661-4dc4-8fad-0a0b4eb06e75 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:52.733516 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe71800-1661-4dc4-8fad-0a0b4eb06e75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:52.735179 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3fe71800-1661-4dc4-8fad-0a0b4eb06e75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:52.802481 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:52.864750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe71800-1661-4dc4-8fad-0a0b4eb06e75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:52.866197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe71800-1661-4dc4-8fad-0a0b4eb06e75 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:52.886435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 655/1319] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:52 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:54:56.592996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b52244-a612-4776-9dc2-1663c548c163 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:56.593644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b52244-a612-4776-9dc2-1663c548c163 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:56.593785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b52244-a612-4776-9dc2-1663c548c163 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:56.595879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 665/1320] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:54:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:54:56.602902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c20d4d-b83d-4778-af3b-affdba2b97e4 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:56.603569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c20d4d-b83d-4778-af3b-affdba2b97e4 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:56.603813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c20d4d-b83d-4778-af3b-affdba2b97e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:56.708585 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-a0c20d4d-b83d-4778-af3b-affdba2b97e4 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:54:56.710616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a0c20d4d-b83d-4778-af3b-affdba2b97e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:56.867103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 656/1321] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:54:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:54:56.910691 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b52244-a612-4776-9dc2-1663c548c163 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:54:56.912738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-83b52244-a612-4776-9dc2-1663c548c163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:56.983072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:57.061400 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b52244-a612-4776-9dc2-1663c548c163 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:54:57.166680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac16856-7420-419a-a127-a3c6830b112c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:54:57.167389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac16856-7420-419a-a127-a3c6830b112c service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:54:57.167757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac16856-7420-419a-a127-a3c6830b112c service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:54:57.168032 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac16856-7420-419a-a127-a3c6830b112c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:54:57.170035 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7ac16856-7420-419a-a127-a3c6830b112c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:54:57.299069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:54:57.316083 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac16856-7420-419a-a127-a3c6830b112c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:54:57.317572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac16856-7420-419a-a127-a3c6830b112c service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:54:57.345754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 666/1322] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:54:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:01.374631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb4e88-11f7-45fc-90c1-e61db6a185bd None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:01.375535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb4e88-11f7-45fc-90c1-e61db6a185bd None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:01.375535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb4e88-11f7-45fc-90c1-e61db6a185bd None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:01.376689 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 657/1323] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:55:01.382701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb2973-60eb-4f1d-b438-06b0b97bc3b8 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:01.383541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb2973-60eb-4f1d-b438-06b0b97bc3b8 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:01.383541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb2973-60eb-4f1d-b438-06b0b97bc3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:01.485159 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-f9bb2973-60eb-4f1d-b438-06b0b97bc3b8 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:01.486995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f9bb2973-60eb-4f1d-b438-06b0b97bc3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:01.645910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 667/1324] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:55:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:01.683903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb4e88-11f7-45fc-90c1-e61db6a185bd None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:01.685776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a3eb4e88-11f7-45fc-90c1-e61db6a185bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:01.713586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:01.851001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb4e88-11f7-45fc-90c1-e61db6a185bd None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:01.974664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b407f-a912-400e-9a3b-0ed652181dde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:01.975279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b407f-a912-400e-9a3b-0ed652181dde service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:01.975707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b407f-a912-400e-9a3b-0ed652181dde service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:01.976050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b407f-a912-400e-9a3b-0ed652181dde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:01.977946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-014b407f-a912-400e-9a3b-0ed652181dde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:02.113882 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:02.177722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b407f-a912-400e-9a3b-0ed652181dde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:02.179311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b407f-a912-400e-9a3b-0ed652181dde service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:02.207361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 658/1325] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:55:04.665721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ed77e-171b-44df-84f2-98736d682263 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:04.666519 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ed77e-171b-44df-84f2-98736d682263 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:04.666723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ed77e-171b-44df-84f2-98736d682263 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:04.669496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 668/1326] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:55:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:55:04.679498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846594e-1467-4880-8679-1e213699adfa None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:04.680397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846594e-1467-4880-8679-1e213699adfa None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:04.680553 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846594e-1467-4880-8679-1e213699adfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:04.806700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-a846594e-1467-4880-8679-1e213699adfa None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:04.808774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a846594e-1467-4880-8679-1e213699adfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:05.020691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 659/1327] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:05.081422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ed77e-171b-44df-84f2-98736d682263 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:05.083142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-af0ed77e-171b-44df-84f2-98736d682263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:05.172106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:05.234513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ed77e-171b-44df-84f2-98736d682263 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:05.371795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc280874-2e40-4fde-8a53-0f3f5a981d9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:05.373188 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc280874-2e40-4fde-8a53-0f3f5a981d9b service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:05.373480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc280874-2e40-4fde-8a53-0f3f5a981d9b service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:05.373896 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc280874-2e40-4fde-8a53-0f3f5a981d9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:05.376920 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-fc280874-2e40-4fde-8a53-0f3f5a981d9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:05.583615 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:05.588024 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc280874-2e40-4fde-8a53-0f3f5a981d9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:05.590051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc280874-2e40-4fde-8a53-0f3f5a981d9b service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:05.622446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 669/1328] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:55:08.217931 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7cabd-2bda-4129-ae13-291fbfbe543a None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:08.218588 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7cabd-2bda-4129-ae13-291fbfbe543a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:08.219000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7cabd-2bda-4129-ae13-291fbfbe543a None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:08.221426 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 660/1329] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:55:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:55:08.227936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c329-ac81-4555-82a8-14550f10d2ee None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:08.228813 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c329-ac81-4555-82a8-14550f10d2ee None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:08.228973 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c329-ac81-4555-82a8-14550f10d2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:08.361208 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-f004c329-ac81-4555-82a8-14550f10d2ee None None] MFA Rules not processed for user `843d8bc8740f48aaa93c0f79dc9687d0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:08.362746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f004c329-ac81-4555-82a8-14550f10d2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:08.518412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 670/1330] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:55:08 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:08.561556 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7cabd-2bda-4129-ae13-291fbfbe543a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:08.563601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1ab7cabd-2bda-4129-ae13-291fbfbe543a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:08.604038 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:08.728794 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7cabd-2bda-4129-ae13-291fbfbe543a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:08.830748 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c1500-b7e9-4074-adfc-0f21e41f1f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:08.831386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c1500-b7e9-4074-adfc-0f21e41f1f30 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:08.831703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c1500-b7e9-4074-adfc-0f21e41f1f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:08.831987 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c1500-b7e9-4074-adfc-0f21e41f1f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:08.833739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-067c1500-b7e9-4074-adfc-0f21e41f1f30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:08.940433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:09.005707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c1500-b7e9-4074-adfc-0f21e41f1f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:09.008636 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c1500-b7e9-4074-adfc-0f21e41f1f30 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:09.049057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 661/1331] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:08 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:55:12.085423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1e531-dc55-41e0-a5ef-9dab2903b62f None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:12.086273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1e531-dc55-41e0-a5ef-9dab2903b62f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:12.086273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1e531-dc55-41e0-a5ef-9dab2903b62f None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:12.087527 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 671/1332] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:55:12.096262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876db781-8673-447e-bfd5-99830b6f5c79 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:12.096840 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876db781-8673-447e-bfd5-99830b6f5c79 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:12.097177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876db781-8673-447e-bfd5-99830b6f5c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:12.226837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-876db781-8673-447e-bfd5-99830b6f5c79 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:12.229087 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-876db781-8673-447e-bfd5-99830b6f5c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:12.425607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 662/1333] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:12.465044 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1e531-dc55-41e0-a5ef-9dab2903b62f None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:12.466641 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-8ae1e531-dc55-41e0-a5ef-9dab2903b62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:12.571924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:12.617864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1e531-dc55-41e0-a5ef-9dab2903b62f None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:12.714814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329da396-bd3d-49f8-b8ba-828dde2281a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:12.715356 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329da396-bd3d-49f8-b8ba-828dde2281a6 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:12.715674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329da396-bd3d-49f8-b8ba-828dde2281a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:12.715947 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329da396-bd3d-49f8-b8ba-828dde2281a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:12.717722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-329da396-bd3d-49f8-b8ba-828dde2281a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:12.851527 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329da396-bd3d-49f8-b8ba-828dde2281a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:12.852980 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329da396-bd3d-49f8-b8ba-828dde2281a6 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:12.865307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:12.874672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 672/1334] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:15.223845 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caefcac-5aee-4a0a-aa8a-020c2e6fc6ec None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:15.224646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caefcac-5aee-4a0a-aa8a-020c2e6fc6ec None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:15.224646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caefcac-5aee-4a0a-aa8a-020c2e6fc6ec None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:15.226457 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 663/1335] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:55:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:55:15.232371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58e0b5-3fb6-4a28-80f7-e5797e759ff7 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:15.233095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58e0b5-3fb6-4a28-80f7-e5797e759ff7 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:15.233095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58e0b5-3fb6-4a28-80f7-e5797e759ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:15.326345 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7f58e0b5-3fb6-4a28-80f7-e5797e759ff7 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:15.327729 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7f58e0b5-3fb6-4a28-80f7-e5797e759ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:15.471963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 673/1336] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:15.519849 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caefcac-5aee-4a0a-aa8a-020c2e6fc6ec None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:15.521501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-6caefcac-5aee-4a0a-aa8a-020c2e6fc6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:15.580979 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:15.683389 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caefcac-5aee-4a0a-aa8a-020c2e6fc6ec None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:15.791651 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4e53-681e-45d8-afa9-81755ebe91b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:15.792301 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4e53-681e-45d8-afa9-81755ebe91b8 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:15.792577 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4e53-681e-45d8-afa9-81755ebe91b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:15.792862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4e53-681e-45d8-afa9-81755ebe91b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:15.794611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-49aa4e53-681e-45d8-afa9-81755ebe91b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:15.905312 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:15.941959 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa4e53-681e-45d8-afa9-81755ebe91b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:15.943698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa4e53-681e-45d8-afa9-81755ebe91b8 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:15.974892 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 664/1337] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:19.143743 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd18d71-e77a-43be-aaf4-4fd688461a24 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:19.144391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd18d71-e77a-43be-aaf4-4fd688461a24 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:19.144391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd18d71-e77a-43be-aaf4-4fd688461a24 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:19.145768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 674/1338] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:55:19.152141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f819a-c5c4-4d9c-b105-76957e1be3f0 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:19.152993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f819a-c5c4-4d9c-b105-76957e1be3f0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:19.153169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f819a-c5c4-4d9c-b105-76957e1be3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:19.245359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-5f5f819a-c5c4-4d9c-b105-76957e1be3f0 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:19.246831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-5f5f819a-c5c4-4d9c-b105-76957e1be3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:19.373639 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 665/1339] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:55:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:19.429204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd18d71-e77a-43be-aaf4-4fd688461a24 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:19.430878 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-bfd18d71-e77a-43be-aaf4-4fd688461a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:19.560227 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:19.596346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd18d71-e77a-43be-aaf4-4fd688461a24 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:19.699739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af780f-ba2c-4205-944f-c36cc5c8977d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:19.700495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af780f-ba2c-4205-944f-c36cc5c8977d service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:19.700818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af780f-ba2c-4205-944f-c36cc5c8977d service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:19.701100 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af780f-ba2c-4205-944f-c36cc5c8977d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:19.702880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e9af780f-ba2c-4205-944f-c36cc5c8977d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:19.842229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af780f-ba2c-4205-944f-c36cc5c8977d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:19.844575 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af780f-ba2c-4205-944f-c36cc5c8977d service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:19.868151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 675/1340] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:55:25.211145 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c5862-3fb2-46a5-8d14-f3ff89ea2aba None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:25.212100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c5862-3fb2-46a5-8d14-f3ff89ea2aba None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:25.212100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c5862-3fb2-46a5-8d14-f3ff89ea2aba None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:25.213570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 666/1341] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:55:25.220208 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d5d5b-b465-4cd0-9c78-63c33a473776 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:25.220935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d5d5b-b465-4cd0-9c78-63c33a473776 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:25.220935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d5d5b-b465-4cd0-9c78-63c33a473776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:25.227413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:25.325655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-bc4d5d5b-b465-4cd0-9c78-63c33a473776 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:25.326933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-bc4d5d5b-b465-4cd0-9c78-63c33a473776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:25.505429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 676/1342] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:25.553387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c5862-3fb2-46a5-8d14-f3ff89ea2aba None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:25.557238 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-623c5862-3fb2-46a5-8d14-f3ff89ea2aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:25.644473 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:25.752390 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c5862-3fb2-46a5-8d14-f3ff89ea2aba None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:25.900285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad398b-3548-49a1-aa30-90b52fbd10ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:25.900932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad398b-3548-49a1-aa30-90b52fbd10ab service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:25.901222 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad398b-3548-49a1-aa30-90b52fbd10ab service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:25.901510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad398b-3548-49a1-aa30-90b52fbd10ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:25.903486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-82ad398b-3548-49a1-aa30-90b52fbd10ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:26.087069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:26.109821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad398b-3548-49a1-aa30-90b52fbd10ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:26.111718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad398b-3548-49a1-aa30-90b52fbd10ab service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:26.136999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 667/1343] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:29.728002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4a631-c40d-43b8-b461-eac38612842b None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:29.728766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4a631-c40d-43b8-b461-eac38612842b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:29.728766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4a631-c40d-43b8-b461-eac38612842b None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:29.730072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 677/1344] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:55:29.737942 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c661f9-97de-482e-b56d-921401419aeb None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:29.738539 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c661f9-97de-482e-b56d-921401419aeb None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:29.738926 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c661f9-97de-482e-b56d-921401419aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:29.833760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-99c661f9-97de-482e-b56d-921401419aeb None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:29.835080 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-99c661f9-97de-482e-b56d-921401419aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:29.951406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 668/1345] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:55:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:30.001203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4a631-c40d-43b8-b461-eac38612842b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:30.004163 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2db4a631-c40d-43b8-b461-eac38612842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:30.144084 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:30.166890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4a631-c40d-43b8-b461-eac38612842b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:30.274051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf42808-b865-44e9-a67a-9de7ecbcfab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:30.274655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf42808-b865-44e9-a67a-9de7ecbcfab8 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:30.274932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf42808-b865-44e9-a67a-9de7ecbcfab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:30.275200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf42808-b865-44e9-a67a-9de7ecbcfab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:30.276915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9cf42808-b865-44e9-a67a-9de7ecbcfab8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:30.418515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf42808-b865-44e9-a67a-9de7ecbcfab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:30.419960 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf42808-b865-44e9-a67a-9de7ecbcfab8 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:30.444334 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 678/1346] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:35.922116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa0078-e78d-4bfb-9452-55b2747015b0 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:35.922675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa0078-e78d-4bfb-9452-55b2747015b0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:35.923254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa0078-e78d-4bfb-9452-55b2747015b0 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:35.926189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 669/1347] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:55:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:55:35.934960 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50208edb-e9f6-4419-9d5c-06bccdae13b4 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:35.935533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50208edb-e9f6-4419-9d5c-06bccdae13b4 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:35.936252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50208edb-e9f6-4419-9d5c-06bccdae13b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:35.965048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:36.033245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-50208edb-e9f6-4419-9d5c-06bccdae13b4 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:36.034579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-50208edb-e9f6-4419-9d5c-06bccdae13b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:36.150207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 679/1348] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:36.194238 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa0078-e78d-4bfb-9452-55b2747015b0 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:36.195859 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-fcaa0078-e78d-4bfb-9452-55b2747015b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:36.278343 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:36.336251 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa0078-e78d-4bfb-9452-55b2747015b0 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:36.437339 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e002ef-9927-4e7e-9395-69a843f6d074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:36.438109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e002ef-9927-4e7e-9395-69a843f6d074 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:36.438339 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e002ef-9927-4e7e-9395-69a843f6d074 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:36.438619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e002ef-9927-4e7e-9395-69a843f6d074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:36.440504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-96e002ef-9927-4e7e-9395-69a843f6d074 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:36.591259 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:36.597811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e002ef-9927-4e7e-9395-69a843f6d074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:36.599323 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e002ef-9927-4e7e-9395-69a843f6d074 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:36.621671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 670/1349] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:42.131298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cec213-e0d1-484c-a799-b3c8ccd68465 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:42.131870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cec213-e0d1-484c-a799-b3c8ccd68465 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:42.131870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cec213-e0d1-484c-a799-b3c8ccd68465 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:42.133442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 680/1350] 10.222.0.94 () {54 vars in 921 bytes} [Tue Jul 26 14:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:55:42.141607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b735ed0-90c3-476d-a478-ecf238325459 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:42.143428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b735ed0-90c3-476d-a478-ecf238325459 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:42.144852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b735ed0-90c3-476d-a478-ecf238325459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:42.321752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-3b735ed0-90c3-476d-a478-ecf238325459 None None] MFA Rules not processed for user `42a0728e2f6249bdb45d6f44fb15aa4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:42.324064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3b735ed0-90c3-476d-a478-ecf238325459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:42.498379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 671/1351] 10.222.0.94 () {58 vars in 1018 bytes} [Tue Jul 26 14:55:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:46.148069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=59 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:48.002932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f89e0-09ae-4be4-8d0f-bf2198bda559 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:48.004140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f89e0-09ae-4be4-8d0f-bf2198bda559 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:48.004595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f89e0-09ae-4be4-8d0f-bf2198bda559 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:48.006454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 681/1352] 10.222.0.94 () {54 vars in 921 bytes} [Tue Jul 26 14:55:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:55:48.012844 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af18690-4bea-483c-b6f7-1ff7fa30480f None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:48.014999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af18690-4bea-483c-b6f7-1ff7fa30480f None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:48.015610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af18690-4bea-483c-b6f7-1ff7fa30480f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:48.129621 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:48.182508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-8af18690-4bea-483c-b6f7-1ff7fa30480f None None] MFA Rules not processed for user `42a0728e2f6249bdb45d6f44fb15aa4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:48.186205 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8af18690-4bea-483c-b6f7-1ff7fa30480f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:48.430400 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 672/1353] 10.222.0.94 () {58 vars in 1018 bytes} [Tue Jul 26 14:55:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:49.737560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226e483-0284-46f8-84c1-4bd51fbeb910 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:49.738416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226e483-0284-46f8-84c1-4bd51fbeb910 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:49.738416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226e483-0284-46f8-84c1-4bd51fbeb910 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:49.739663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 682/1354] 10.222.0.94 () {54 vars in 921 bytes} [Tue Jul 26 14:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:55:49.746765 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec1412-f9dd-4d87-9b41-142003b88e41 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:49.747392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec1412-f9dd-4d87-9b41-142003b88e41 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:49.747871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec1412-f9dd-4d87-9b41-142003b88e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:49.867082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-cbec1412-f9dd-4d87-9b41-142003b88e41 None None] MFA Rules not processed for user `42a0728e2f6249bdb45d6f44fb15aa4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:49.868452 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-cbec1412-f9dd-4d87-9b41-142003b88e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:49.941137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:50.034596 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 673/1355] 10.222.0.94 () {58 vars in 1018 bytes} [Tue Jul 26 14:55:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:54.917344 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0338c91-e0a9-43d3-9c8a-9b4ead58eef6 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:54.918148 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0338c91-e0a9-43d3-9c8a-9b4ead58eef6 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:54.918148 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0338c91-e0a9-43d3-9c8a-9b4ead58eef6 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:54.919253 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 683/1356] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:55:54.930338 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1761920-ca33-4738-b4e5-411ba5598d0f None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:54.930874 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1761920-ca33-4738-b4e5-411ba5598d0f None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:54.930874 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1761920-ca33-4738-b4e5-411ba5598d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:55.027414 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-b1761920-ca33-4738-b4e5-411ba5598d0f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:55.029009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b1761920-ca33-4738-b4e5-411ba5598d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:55.189944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 674/1357] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:55.291893 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37932618-7bf1-42d0-b85c-23b12f0a8889 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:55.292523 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37932618-7bf1-42d0-b85c-23b12f0a8889 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:55.293398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37932618-7bf1-42d0-b85c-23b12f0a8889 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:55.296737 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 684/1358] 10.222.0.94 () {54 vars in 962 bytes} [Tue Jul 26 14:55:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:55:55.312823 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42028cc7-090b-48a1-b1f1-c035ef8d8dab None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:55.313770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42028cc7-090b-48a1-b1f1-c035ef8d8dab None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:55.314352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42028cc7-090b-48a1-b1f1-c035ef8d8dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:55.339942 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:55.445479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-42028cc7-090b-48a1-b1f1-c035ef8d8dab None None] MFA Rules not processed for user `6a4278fe995e432d88f4346d6f54218b`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:55.446850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-42028cc7-090b-48a1-b1f1-c035ef8d8dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:55.622914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 675/1359] 10.222.0.94 () {58 vars in 1059 bytes} [Tue Jul 26 14:55:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:55:55.643915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37932618-7bf1-42d0-b85c-23b12f0a8889 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:55.645492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-37932618-7bf1-42d0-b85c-23b12f0a8889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:55.835676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37932618-7bf1-42d0-b85c-23b12f0a8889 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:55.928595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:55.980068 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209932c-ab26-46f3-8ce9-51e13b2cd150 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:55.980697 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209932c-ab26-46f3-8ce9-51e13b2cd150 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:55.980979 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209932c-ab26-46f3-8ce9-51e13b2cd150 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:55.981254 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209932c-ab26-46f3-8ce9-51e13b2cd150 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:55.983630 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-8209932c-ab26-46f3-8ce9-51e13b2cd150 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:56.105767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209932c-ab26-46f3-8ce9-51e13b2cd150 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:55:56.108490 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209932c-ab26-46f3-8ce9-51e13b2cd150 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:55:56.131743 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 685/1360] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:55:59.150994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c9060-4816-4316-9fb6-13c5a2c5fb71 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:59.151994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c9060-4816-4316-9fb6-13c5a2c5fb71 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:59.151994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c9060-4816-4316-9fb6-13c5a2c5fb71 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:59.154345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 676/1361] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:55:59.159764 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd295746-6e50-40ea-9b38-6ff7da59a69b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:59.160564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd295746-6e50-40ea-9b38-6ff7da59a69b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:59.160564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd295746-6e50-40ea-9b38-6ff7da59a69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:59.306734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-cd295746-6e50-40ea-9b38-6ff7da59a69b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:55:59.312782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-cd295746-6e50-40ea-9b38-6ff7da59a69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:59.363506 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:59.458778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 686/1362] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:55:59.526604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c9060-4816-4316-9fb6-13c5a2c5fb71 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:55:59.530492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-208c9060-4816-4316-9fb6-13c5a2c5fb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:55:59.663911 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:55:59.791742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c9060-4816-4316-9fb6-13c5a2c5fb71 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:55:59.976683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ef54d-dc4a-4d70-99be-22eaaf5de68f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:55:59.977442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ef54d-dc4a-4d70-99be-22eaaf5de68f service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:55:59.978184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ef54d-dc4a-4d70-99be-22eaaf5de68f service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:55:59.978691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ef54d-dc4a-4d70-99be-22eaaf5de68f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:55:59.981000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-478ef54d-dc4a-4d70-99be-22eaaf5de68f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:00.026295 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:00.181735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ef54d-dc4a-4d70-99be-22eaaf5de68f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:00.184941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ef54d-dc4a-4d70-99be-22eaaf5de68f service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:00.240663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 677/1363] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:55:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:56:03.064454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452db0e-508d-4ada-99a7-809d513b8a4d None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:03.065065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452db0e-508d-4ada-99a7-809d513b8a4d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:03.065065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452db0e-508d-4ada-99a7-809d513b8a4d None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:03.070210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 687/1364] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:03.072646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec4e16-b8be-4ebe-bf02-3d93d22099c2 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:03.073166 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec4e16-b8be-4ebe-bf02-3d93d22099c2 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:03.073411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec4e16-b8be-4ebe-bf02-3d93d22099c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:03.201450 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-20ec4e16-b8be-4ebe-bf02-3d93d22099c2 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:03.203223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-20ec4e16-b8be-4ebe-bf02-3d93d22099c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:03.365429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:03.371556 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 678/1365] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:03.451938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c1ee3-0a8d-48df-84e8-bc445d4aceaf None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:03.452599 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c1ee3-0a8d-48df-84e8-bc445d4aceaf None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:03.453229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c1ee3-0a8d-48df-84e8-bc445d4aceaf None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:03.456300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 688/1366] 10.222.0.94 () {54 vars in 962 bytes} [Tue Jul 26 14:56:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:03.466413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13e448-7bb9-43cd-ad38-e871e6635f32 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:03.466939 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13e448-7bb9-43cd-ad38-e871e6635f32 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:03.467756 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13e448-7bb9-43cd-ad38-e871e6635f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:03.627448 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-3c13e448-7bb9-43cd-ad38-e871e6635f32 None None] MFA Rules not processed for user `6a4278fe995e432d88f4346d6f54218b`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:03.630135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3c13e448-7bb9-43cd-ad38-e871e6635f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:03.844872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 679/1367] 10.222.0.94 () {58 vars in 1059 bytes} [Tue Jul 26 14:56:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:03.855726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c1ee3-0a8d-48df-84e8-bc445d4aceaf None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:03.858656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-712c1ee3-0a8d-48df-84e8-bc445d4aceaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:04.104471 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c1ee3-0a8d-48df-84e8-bc445d4aceaf None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:04.140202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:04.281479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f85d8c-399c-43d3-bfd8-d0ce8c3695e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:04.282248 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f85d8c-399c-43d3-bfd8-d0ce8c3695e6 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:04.282585 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f85d8c-399c-43d3-bfd8-d0ce8c3695e6 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:04.282921 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f85d8c-399c-43d3-bfd8-d0ce8c3695e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:04.288071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-22f85d8c-399c-43d3-bfd8-d0ce8c3695e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:04.402101 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f85d8c-399c-43d3-bfd8-d0ce8c3695e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:04.403878 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f85d8c-399c-43d3-bfd8-d0ce8c3695e6 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:04.428676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 689/1368] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:56:04.784995 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89789437-89e1-4619-8edf-72951ec66a90 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:04.790863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89789437-89e1-4619-8edf-72951ec66a90 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:04.790863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89789437-89e1-4619-8edf-72951ec66a90 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:04.790863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 680/1369] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:56:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:04.794540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c39c0-a83c-471b-957a-dc2035b2f8a8 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:04.795913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c39c0-a83c-471b-957a-dc2035b2f8a8 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:04.796430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c39c0-a83c-471b-957a-dc2035b2f8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:04.853543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:04.913828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-1a2c39c0-a83c-471b-957a-dc2035b2f8a8 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:04.915764 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1a2c39c0-a83c-471b-957a-dc2035b2f8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:05.090754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 690/1370] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:05.615923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e315524-d74e-4c56-8612-cddb8b32e784 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:05.616807 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e315524-d74e-4c56-8612-cddb8b32e784 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:05.617379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e315524-d74e-4c56-8612-cddb8b32e784 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:05.620655 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 681/1371] 10.222.0.94 () {54 vars in 964 bytes} [Tue Jul 26 14:56:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:05.627812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c11af-6a0a-4beb-acb6-f581ac307c15 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:05.628865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c11af-6a0a-4beb-acb6-f581ac307c15 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:05.629279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c11af-6a0a-4beb-acb6-f581ac307c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:05.740046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b65c11af-6a0a-4beb-acb6-f581ac307c15 None None] MFA Rules not processed for user `7448959048f746a0ae1588ff839deb0c`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:05.741279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b65c11af-6a0a-4beb-acb6-f581ac307c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:05.807593 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:05.852659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 691/1372] 10.222.0.94 () {58 vars in 1061 bytes} [Tue Jul 26 14:56:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:56:05.867808 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e315524-d74e-4c56-8612-cddb8b32e784 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:05.871031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-5e315524-d74e-4c56-8612-cddb8b32e784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:05.960050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:06.040493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e315524-d74e-4c56-8612-cddb8b32e784 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:06.146009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83904d9-8aee-49b1-9897-dadc308fe256 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:06.146641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83904d9-8aee-49b1-9897-dadc308fe256 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:06.146929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83904d9-8aee-49b1-9897-dadc308fe256 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:06.147206 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83904d9-8aee-49b1-9897-dadc308fe256 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:06.149122 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d83904d9-8aee-49b1-9897-dadc308fe256 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:06.280590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:06.303753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83904d9-8aee-49b1-9897-dadc308fe256 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:06.305598 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83904d9-8aee-49b1-9897-dadc308fe256 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:06.338513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 682/1373] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:56:07.296409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2467ae-613e-4d7e-8a63-79730cfb6b31 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:07.297432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2467ae-613e-4d7e-8a63-79730cfb6b31 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:07.297722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2467ae-613e-4d7e-8a63-79730cfb6b31 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:07.299266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 692/1374] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:56:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:07.305485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3632c-ebd7-4bc2-a4d9-5fe7c946b2de None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:07.306314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3632c-ebd7-4bc2-a4d9-5fe7c946b2de None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:07.306314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3632c-ebd7-4bc2-a4d9-5fe7c946b2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:07.424923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-6dd3632c-ebd7-4bc2-a4d9-5fe7c946b2de None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:07.426946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-6dd3632c-ebd7-4bc2-a4d9-5fe7c946b2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:07.567666 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 683/1375] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:07.616529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2467ae-613e-4d7e-8a63-79730cfb6b31 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:07.618074 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-cb2467ae-613e-4d7e-8a63-79730cfb6b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:07.863976 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2467ae-613e-4d7e-8a63-79730cfb6b31 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:07.949541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:08.055342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9892e5-b4e7-481b-8b84-66905ef9b584 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:08.056466 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9892e5-b4e7-481b-8b84-66905ef9b584 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:08.056936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9892e5-b4e7-481b-8b84-66905ef9b584 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:08.057428 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9892e5-b4e7-481b-8b84-66905ef9b584 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:08.060857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-be9892e5-b4e7-481b-8b84-66905ef9b584 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:08.204080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9892e5-b4e7-481b-8b84-66905ef9b584 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:08.205914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9892e5-b4e7-481b-8b84-66905ef9b584 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:08.238646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 693/1376] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:56:10.710016 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91fc5b-ce89-43b7-aff4-ef6d593e1c15 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:10.710933 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91fc5b-ce89-43b7-aff4-ef6d593e1c15 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:10.710933 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91fc5b-ce89-43b7-aff4-ef6d593e1c15 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:10.712248 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 684/1377] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:56:10.718678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b43c27-098a-4262-9497-0a859e5b7703 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:10.719264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b43c27-098a-4262-9497-0a859e5b7703 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:10.719264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b43c27-098a-4262-9497-0a859e5b7703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:10.828648 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-47b43c27-098a-4262-9497-0a859e5b7703 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:10.829956 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-47b43c27-098a-4262-9497-0a859e5b7703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:10.839794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:11.046518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 694/1378] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:11.100087 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91fc5b-ce89-43b7-aff4-ef6d593e1c15 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:11.102106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-df91fc5b-ce89-43b7-aff4-ef6d593e1c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:11.240121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:11.381018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91fc5b-ce89-43b7-aff4-ef6d593e1c15 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:11.599214 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae965e4-9789-48b7-b455-1c58a5b105c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:11.599946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae965e4-9789-48b7-b455-1c58a5b105c6 service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:11.600229 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae965e4-9789-48b7-b455-1c58a5b105c6 service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:11.600614 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae965e4-9789-48b7-b455-1c58a5b105c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:11.602696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4ae965e4-9789-48b7-b455-1c58a5b105c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:11.629479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:11.765050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae965e4-9789-48b7-b455-1c58a5b105c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:11.768024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae965e4-9789-48b7-b455-1c58a5b105c6 service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:11.808325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 685/1379] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 709 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:56:14.506586 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef6f47-9b44-41a9-a911-f8ba0e776074 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:14.507157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef6f47-9b44-41a9-a911-f8ba0e776074 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:14.507157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef6f47-9b44-41a9-a911-f8ba0e776074 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:14.508869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 695/1380] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:14.515692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129ce4b-37ed-4a80-bb23-66e26113aa45 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:14.516461 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129ce4b-37ed-4a80-bb23-66e26113aa45 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:14.516461 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129ce4b-37ed-4a80-bb23-66e26113aa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:14.617553 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-3129ce4b-37ed-4a80-bb23-66e26113aa45 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:14.619042 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3129ce4b-37ed-4a80-bb23-66e26113aa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:14.752023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:14.759587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 686/1381] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:14.811322 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef6f47-9b44-41a9-a911-f8ba0e776074 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:14.814126 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e9ef6f47-9b44-41a9-a911-f8ba0e776074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:15.049371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:15.062991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef6f47-9b44-41a9-a911-f8ba0e776074 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:15.212588 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d4b51-3ae3-4493-9f34-803dd1f9c3e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:15.213590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d4b51-3ae3-4493-9f34-803dd1f9c3e3 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:15.213778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d4b51-3ae3-4493-9f34-803dd1f9c3e3 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:15.214329 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d4b51-3ae3-4493-9f34-803dd1f9c3e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:15.216876 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d40d4b51-3ae3-4493-9f34-803dd1f9c3e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:15.361540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d4b51-3ae3-4493-9f34-803dd1f9c3e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:15.363607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d4b51-3ae3-4493-9f34-803dd1f9c3e3 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:15.378141 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:15.390692 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 696/1382] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:56:18.012199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec57a2-2981-493b-84ff-47fae95af1e0 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:18.013586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec57a2-2981-493b-84ff-47fae95af1e0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:18.013586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec57a2-2981-493b-84ff-47fae95af1e0 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:18.014808 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 687/1383] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:56:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:56:18.026236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc435a4-4458-4816-bea9-72c6289dcdfc None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:18.026794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc435a4-4458-4816-bea9-72c6289dcdfc None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:18.026794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc435a4-4458-4816-bea9-72c6289dcdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:18.111306 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-2dc435a4-4458-4816-bea9-72c6289dcdfc None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:18.112579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2dc435a4-4458-4816-bea9-72c6289dcdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:18.259190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 697/1384] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:18.319412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec57a2-2981-493b-84ff-47fae95af1e0 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:18.324310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-cbec57a2-2981-493b-84ff-47fae95af1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:18.542171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec57a2-2981-493b-84ff-47fae95af1e0 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:18.672943 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:18.753435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e520944-e464-426a-a8b0-806947d329f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:18.754452 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e520944-e464-426a-a8b0-806947d329f8 service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:18.754990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e520944-e464-426a-a8b0-806947d329f8 service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:18.755443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e520944-e464-426a-a8b0-806947d329f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:18.759210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4e520944-e464-426a-a8b0-806947d329f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:18.885255 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e520944-e464-426a-a8b0-806947d329f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:18.887315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e520944-e464-426a-a8b0-806947d329f8 service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:18.910814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 688/1385] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:56:21.353514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf881-8f20-4878-b50f-b5d8fe78c14b None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:21.354294 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf881-8f20-4878-b50f-b5d8fe78c14b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:21.354294 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf881-8f20-4878-b50f-b5d8fe78c14b None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:21.355414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 698/1386] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:21.364157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f829fae-4256-494c-96f5-523fdec49533 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:21.365033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f829fae-4256-494c-96f5-523fdec49533 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:21.365033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f829fae-4256-494c-96f5-523fdec49533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:21.472659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-9f829fae-4256-494c-96f5-523fdec49533 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:21.474058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9f829fae-4256-494c-96f5-523fdec49533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:21.529125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:21.662730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 689/1387] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:21.725824 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcf881-8f20-4878-b50f-b5d8fe78c14b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:21.727638 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-0abcf881-8f20-4878-b50f-b5d8fe78c14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:21.848155 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:21.949410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcf881-8f20-4878-b50f-b5d8fe78c14b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:22.151959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32854ce0-ca1d-48c4-86e4-b34daa972f6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:22.152563 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32854ce0-ca1d-48c4-86e4-b34daa972f6c service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:22.152844 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32854ce0-ca1d-48c4-86e4-b34daa972f6c service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:22.153114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32854ce0-ca1d-48c4-86e4-b34daa972f6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:22.155313 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-32854ce0-ca1d-48c4-86e4-b34daa972f6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:22.216882 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:22.303316 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32854ce0-ca1d-48c4-86e4-b34daa972f6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:22.305767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32854ce0-ca1d-48c4-86e4-b34daa972f6c service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:22.339862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 699/1388] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:56:24.826291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8042855-d3e7-4d7c-80c8-9316d76327fa None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:24.827040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8042855-d3e7-4d7c-80c8-9316d76327fa None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:24.827040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8042855-d3e7-4d7c-80c8-9316d76327fa None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:24.828427 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 690/1389] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:24.835736 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63f0e9-3036-4244-9c53-7279a30501f7 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:24.836298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63f0e9-3036-4244-9c53-7279a30501f7 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:24.836595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63f0e9-3036-4244-9c53-7279a30501f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:24.949614 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-aa63f0e9-3036-4244-9c53-7279a30501f7 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:24.951730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-aa63f0e9-3036-4244-9c53-7279a30501f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:25.088866 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 700/1390] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:25.140073 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8042855-d3e7-4d7c-80c8-9316d76327fa None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:25.143369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b8042855-d3e7-4d7c-80c8-9316d76327fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:25.390676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8042855-d3e7-4d7c-80c8-9316d76327fa None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:25.406807 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193a79d-a3b6-4bb7-b40b-07678ff7c31e None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:25.407111 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193a79d-a3b6-4bb7-b40b-07678ff7c31e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:25.407482 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193a79d-a3b6-4bb7-b40b-07678ff7c31e None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:25.408974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 701/1391] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:25.414514 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:25.415978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ce588-1c27-478e-b425-2f847be3df94 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:25.416335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ce588-1c27-478e-b425-2f847be3df94 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:25.416629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ce588-1c27-478e-b425-2f847be3df94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:25.439331 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:25.529038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-bb0ce588-1c27-478e-b425-2f847be3df94 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:25.530399 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-bb0ce588-1c27-478e-b425-2f847be3df94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:25.574680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfca4c1-f1c0-47f9-bf82-cb42f5d02d76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:25.576211 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfca4c1-f1c0-47f9-bf82-cb42f5d02d76 service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:25.581600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfca4c1-f1c0-47f9-bf82-cb42f5d02d76 service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:25.581600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfca4c1-f1c0-47f9-bf82-cb42f5d02d76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:25.581600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-abfca4c1-f1c0-47f9-bf82-cb42f5d02d76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:25.668054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 702/1392] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:25.727145 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfca4c1-f1c0-47f9-bf82-cb42f5d02d76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:25.729201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfca4c1-f1c0-47f9-bf82-cb42f5d02d76 service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:25.735374 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73715c6-0a55-407c-8bfd-2ea319f6d888 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:25.737724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73715c6-0a55-407c-8bfd-2ea319f6d888 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:25.737724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73715c6-0a55-407c-8bfd-2ea319f6d888 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:25.737724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 703/1393] 10.222.0.94 () {54 vars in 966 bytes} [Tue Jul 26 14:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:25.743962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2547f23-80b7-43aa-bc18-e9e76fb0e98d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:25.744364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2547f23-80b7-43aa-bc18-e9e76fb0e98d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:25.744626 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2547f23-80b7-43aa-bc18-e9e76fb0e98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:25.756660 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 691/1394] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:56:25.841730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b2547f23-80b7-43aa-bc18-e9e76fb0e98d None None] MFA Rules not processed for user `b153935910c142aa80792e111872318a`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:25.842509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b2547f23-80b7-43aa-bc18-e9e76fb0e98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:25.896381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:25.945175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 704/1395] 10.222.0.94 () {58 vars in 1063 bytes} [Tue Jul 26 14:56:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:56:25.953473 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfca4c1-f1c0-47f9-bf82-cb42f5d02d76 service nova] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:25.954792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-abfca4c1-f1c0-47f9-bf82-cb42f5d02d76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:25.990390 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:26.077373 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfca4c1-f1c0-47f9-bf82-cb42f5d02d76 service nova] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:26.178237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8c7ce-03ff-481e-82c3-fc4cd5dfc707 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b153935910c142aa80792e111872318a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:26.178879 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8c7ce-03ff-481e-82c3-fc4cd5dfc707 service glance] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:26.179156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8c7ce-03ff-481e-82c3-fc4cd5dfc707 service glance] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:26.179433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8c7ce-03ff-481e-82c3-fc4cd5dfc707 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:26.181350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-11a8c7ce-03ff-481e-82c3-fc4cd5dfc707 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:26.360293 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:26.387394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8c7ce-03ff-481e-82c3-fc4cd5dfc707 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:26.390337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8c7ce-03ff-481e-82c3-fc4cd5dfc707 service glance] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:26.429604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 692/1396] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:56:27.532064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a438827-0c1f-4806-95e6-0c973f6f54a0 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:27.532927 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a438827-0c1f-4806-95e6-0c973f6f54a0 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:27.532927 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a438827-0c1f-4806-95e6-0c973f6f54a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:27.624923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-8a438827-0c1f-4806-95e6-0c973f6f54a0 None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:27.626313 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-8a438827-0c1f-4806-95e6-0c973f6f54a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:27.756636 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 705/1397] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 14:56:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:27.840468 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2ed07-10ab-4c25-a72d-6cd5a5eba498 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:27.840468 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2ed07-10ab-4c25-a72d-6cd5a5eba498 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:27.840468 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2ed07-10ab-4c25-a72d-6cd5a5eba498 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:27.840468 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 693/1398] 10.222.0.94 () {54 vars in 943 bytes} [Tue Jul 26 14:56:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:27.843884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd31a8-7721-480c-9e80-85527077f939 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:27.843884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd31a8-7721-480c-9e80-85527077f939 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:27.843884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd31a8-7721-480c-9e80-85527077f939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:27.913382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:27.939062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-90bd31a8-7721-480c-9e80-85527077f939 None None] MFA Rules not processed for user `b6cea7b5ba4a4acabe8a26837cd97310`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:27.940705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-90bd31a8-7721-480c-9e80-85527077f939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:28.159891 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 706/1399] 10.222.0.94 () {58 vars in 1040 bytes} [Tue Jul 26 14:56:27 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:56:28.176999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2ed07-10ab-4c25-a72d-6cd5a5eba498 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:28.180313 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4ab2ed07-10ab-4c25-a72d-6cd5a5eba498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:28.389231 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359cf9d-dc37-440f-b032-749f6355f065 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:28.389567 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359cf9d-dc37-440f-b032-749f6355f065 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:28.389779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359cf9d-dc37-440f-b032-749f6355f065 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:28.391334 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 707/1400] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:56:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 14:56:28.397668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1098d-ec74-4e3f-a3c9-61c4fbb5fe58 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:28.397965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1098d-ec74-4e3f-a3c9-61c4fbb5fe58 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:28.398576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1098d-ec74-4e3f-a3c9-61c4fbb5fe58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:28.469224 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2ed07-10ab-4c25-a72d-6cd5a5eba498 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:28.491689 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-8dc1098d-ec74-4e3f-a3c9-61c4fbb5fe58 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:28.494251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-8dc1098d-ec74-4e3f-a3c9-61c4fbb5fe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:28.576919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:28.578917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:28.645627 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 708/1401] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:56:28.646449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89d404-4d93-4110-8b85-e4a0b8df675a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:28.647501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89d404-4d93-4110-8b85-e4a0b8df675a service swift] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:28.648001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89d404-4d93-4110-8b85-e4a0b8df675a service swift] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:28.648544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89d404-4d93-4110-8b85-e4a0b8df675a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:28.651921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1e89d404-4d93-4110-8b85-e4a0b8df675a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:28.700073 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1098d-ec74-4e3f-a3c9-61c4fbb5fe58 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:28.702384 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-8dc1098d-ec74-4e3f-a3c9-61c4fbb5fe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:28.789097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89d404-4d93-4110-8b85-e4a0b8df675a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:28.792265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89d404-4d93-4110-8b85-e4a0b8df675a service swift] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:28.808268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 694/1402] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 14:56:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 14:56:28.893146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1098d-ec74-4e3f-a3c9-61c4fbb5fe58 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:28.993048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:29.056791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cec7ca-440f-4f33-8509-474962820b65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:29.057315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cec7ca-440f-4f33-8509-474962820b65 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:29.057595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cec7ca-440f-4f33-8509-474962820b65 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:29.057865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cec7ca-440f-4f33-8509-474962820b65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:29.059729 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e2cec7ca-440f-4f33-8509-474962820b65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:29.196263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cec7ca-440f-4f33-8509-474962820b65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:29.199402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cec7ca-440f-4f33-8509-474962820b65 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:29.230662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 709/1403] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:56:31.820436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e122c29-4a2e-4880-8641-135d32985e3e None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:31.821346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e122c29-4a2e-4880-8641-135d32985e3e None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:31.821909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e122c29-4a2e-4880-8641-135d32985e3e None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:31.824120 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 695/1404] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:56:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:31.830571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8ea54-6520-4e6b-a41d-cf13137c515e None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:31.831149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8ea54-6520-4e6b-a41d-cf13137c515e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:31.831247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8ea54-6520-4e6b-a41d-cf13137c515e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:31.957394 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-a9d8ea54-6520-4e6b-a41d-cf13137c515e None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:31.958688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a9d8ea54-6520-4e6b-a41d-cf13137c515e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:32.016537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:32.131082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 710/1405] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:56:32.193320 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e122c29-4a2e-4880-8641-135d32985e3e None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:32.195297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-2e122c29-4a2e-4880-8641-135d32985e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:32.356990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:32.427401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e122c29-4a2e-4880-8641-135d32985e3e None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:32.642113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbc8ef-dcbc-4af2-a08d-bbd1959747ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:32.643022 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbc8ef-dcbc-4af2-a08d-bbd1959747ea service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:32.643484 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbc8ef-dcbc-4af2-a08d-bbd1959747ea service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:32.643896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbc8ef-dcbc-4af2-a08d-bbd1959747ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:32.646723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-04bbc8ef-dcbc-4af2-a08d-bbd1959747ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:32.787281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:32.830862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbc8ef-dcbc-4af2-a08d-bbd1959747ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:32.833202 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbc8ef-dcbc-4af2-a08d-bbd1959747ea service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:32.862842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 696/1406] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:56:35.621932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2f7ba-9f41-4cc0-9747-716f90c48c31 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:35.622599 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2f7ba-9f41-4cc0-9747-716f90c48c31 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:35.622927 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2f7ba-9f41-4cc0-9747-716f90c48c31 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:35.625630 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 711/1407] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:56:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:35.633072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371c519-1ae1-4c5f-9395-2a53bec37e64 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:35.633657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371c519-1ae1-4c5f-9395-2a53bec37e64 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:35.633657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371c519-1ae1-4c5f-9395-2a53bec37e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:35.724328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-5371c519-1ae1-4c5f-9395-2a53bec37e64 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:35.725791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-5371c519-1ae1-4c5f-9395-2a53bec37e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:35.872957 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 697/1408] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:35.958736 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2f7ba-9f41-4cc0-9747-716f90c48c31 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:56:35.961556 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c9c2f7ba-9f41-4cc0-9747-716f90c48c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:36.197982 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2f7ba-9f41-4cc0-9747-716f90c48c31 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:56:36.222510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:36.405991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e9e73-15c5-41b8-9d2b-4105aff3f872 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:56:36.406985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e9e73-15c5-41b8-9d2b-4105aff3f872 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:36.407357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e9e73-15c5-41b8-9d2b-4105aff3f872 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:36.407781 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e9e73-15c5-41b8-9d2b-4105aff3f872 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:36.410857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7d8e9e73-15c5-41b8-9d2b-4105aff3f872 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:36.583023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e9e73-15c5-41b8-9d2b-4105aff3f872 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:56:36.584965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e9e73-15c5-41b8-9d2b-4105aff3f872 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:56:36.616062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 712/1409] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:56:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:56:45.865590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=81 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:46.608604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:56.166781 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c44f6-3b43-4815-af36-d4332d8ffd8b None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:56.167541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c44f6-3b43-4815-af36-d4332d8ffd8b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:56.167541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c44f6-3b43-4815-af36-d4332d8ffd8b None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:56.169506 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 698/1410] 10.222.0.94 () {54 vars in 918 bytes} [Tue Jul 26 14:56:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:56.175629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f95e0b-42ea-4adc-a5af-a0e68d6aecbd None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:56.176499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f95e0b-42ea-4adc-a5af-a0e68d6aecbd None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:56.176499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f95e0b-42ea-4adc-a5af-a0e68d6aecbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:56.290602 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-d2f95e0b-42ea-4adc-a5af-a0e68d6aecbd None None] MFA Rules not processed for user `42a0728e2f6249bdb45d6f44fb15aa4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:56.293355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d2f95e0b-42ea-4adc-a5af-a0e68d6aecbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:56.447024 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 713/1411] 10.222.0.94 () {58 vars in 1015 bytes} [Tue Jul 26 14:56:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:56:56.470046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a60d4b-36af-4dab-bbd3-ae59f6e99071 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:56.470373 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a60d4b-36af-4dab-bbd3-ae59f6e99071 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:56.470654 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a60d4b-36af-4dab-bbd3-ae59f6e99071 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:56.472558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 699/1412] 10.222.0.94 () {54 vars in 918 bytes} [Tue Jul 26 14:56:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:56.480741 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb41704-0ee9-42e7-af76-78399a1d8436 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:56.481131 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb41704-0ee9-42e7-af76-78399a1d8436 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:56.481439 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb41704-0ee9-42e7-af76-78399a1d8436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:56.587248 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:56:56.592260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-afb41704-0ee9-42e7-af76-78399a1d8436 None None] MFA Rules not processed for user `6a4278fe995e432d88f4346d6f54218b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:56:56.593835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-afb41704-0ee9-42e7-af76-78399a1d8436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:56:56.773254 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 714/1413] 10.222.0.94 () {58 vars in 1015 bytes} [Tue Jul 26 14:56:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:56:59.858827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e2760-7db3-4b2f-ae97-d960fccaa442 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:59.859556 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e2760-7db3-4b2f-ae97-d960fccaa442 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:59.859556 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e2760-7db3-4b2f-ae97-d960fccaa442 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:56:59.860803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:56:59.868001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe0dca-b025-4ccf-9c08-52229376551b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:56:59.868709 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe0dca-b025-4ccf-9c08-52229376551b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:56:59.868709 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe0dca-b025-4ccf-9c08-52229376551b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:00.016430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-0dbe0dca-b025-4ccf-9c08-52229376551b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:00.018644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-0dbe0dca-b025-4ccf-9c08-52229376551b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:00.099955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:00.152550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:00.202680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e2760-7db3-4b2f-ae97-d960fccaa442 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:00.204431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9a4e2760-7db3-4b2f-ae97-d960fccaa442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:00.409478 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e2760-7db3-4b2f-ae97-d960fccaa442 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:00.502950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:00.557558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becaad28-f09f-4496-9e06-2d346c0ff65b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:00.558409 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaad28-f09f-4496-9e06-2d346c0ff65b service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:00.558589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaad28-f09f-4496-9e06-2d346c0ff65b service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:00.558795 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaad28-f09f-4496-9e06-2d346c0ff65b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:00.560541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-becaad28-f09f-4496-9e06-2d346c0ff65b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:00.666015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becaad28-f09f-4496-9e06-2d346c0ff65b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:00.667765 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becaad28-f09f-4496-9e06-2d346c0ff65b service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:00.688445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 701/1416] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:01.685477 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcce9b5-0bb6-49d0-be86-7181d8141b55 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:01.686277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcce9b5-0bb6-49d0-be86-7181d8141b55 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:01.686277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcce9b5-0bb6-49d0-be86-7181d8141b55 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:01.687613 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:57:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:01.693580 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b1af6-b18f-4e83-8e2d-1ff6a91de432 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:01.694133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b1af6-b18f-4e83-8e2d-1ff6a91de432 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:01.694133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b1af6-b18f-4e83-8e2d-1ff6a91de432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:01.790478 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-068b1af6-b18f-4e83-8e2d-1ff6a91de432 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:01.791748 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-068b1af6-b18f-4e83-8e2d-1ff6a91de432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:01.825262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:01.911268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:02.068899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcce9b5-0bb6-49d0-be86-7181d8141b55 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:02.071067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7dcce9b5-0bb6-49d0-be86-7181d8141b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:02.241155 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcce9b5-0bb6-49d0-be86-7181d8141b55 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:02.293463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:02.388391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e6528-e2ec-4b24-9d85-a824fed28a3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:02.388901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e6528-e2ec-4b24-9d85-a824fed28a3a service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:02.389174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e6528-e2ec-4b24-9d85-a824fed28a3a service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:02.389448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e6528-e2ec-4b24-9d85-a824fed28a3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:02.391126 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-782e6528-e2ec-4b24-9d85-a824fed28a3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:02.497352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e6528-e2ec-4b24-9d85-a824fed28a3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:02.499035 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e6528-e2ec-4b24-9d85-a824fed28a3a service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:02.520190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 717/1419] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:03.721924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744c158-114d-4af6-a82f-2da4fe799dd1 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:03.722570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744c158-114d-4af6-a82f-2da4fe799dd1 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:03.722686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744c158-114d-4af6-a82f-2da4fe799dd1 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:03.724140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:03.729864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1f3e0-f6f4-4143-8762-2fcd688369fa None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:03.730897 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1f3e0-f6f4-4143-8762-2fcd688369fa None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:03.731252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1f3e0-f6f4-4143-8762-2fcd688369fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:03.813444 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:03.851226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b9f1f3e0-f6f4-4143-8762-2fcd688369fa None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:03.852493 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b9f1f3e0-f6f4-4143-8762-2fcd688369fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:03.978866 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:04.141089 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744c158-114d-4af6-a82f-2da4fe799dd1 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:04.142644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0744c158-114d-4af6-a82f-2da4fe799dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:04.324351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744c158-114d-4af6-a82f-2da4fe799dd1 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:04.338733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:04.463185 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa6e5b-1bb3-4b15-9b0d-beb227647eae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:04.463852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa6e5b-1bb3-4b15-9b0d-beb227647eae service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:04.464238 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa6e5b-1bb3-4b15-9b0d-beb227647eae service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:04.464553 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa6e5b-1bb3-4b15-9b0d-beb227647eae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:04.466491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-2eaa6e5b-1bb3-4b15-9b0d-beb227647eae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:04.604453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa6e5b-1bb3-4b15-9b0d-beb227647eae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:04.606689 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa6e5b-1bb3-4b15-9b0d-beb227647eae service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:04.626775 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 704/1422] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:05.565499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b7601-daf1-4411-af44-0c03bf2b2366 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:05.566397 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b7601-daf1-4411-af44-0c03bf2b2366 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:05.566397 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b7601-daf1-4411-af44-0c03bf2b2366 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:05.568012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:57:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:05.575848 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afca38-2742-4e1b-a159-1dcb8bf53981 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:05.576619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afca38-2742-4e1b-a159-1dcb8bf53981 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:05.576619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afca38-2742-4e1b-a159-1dcb8bf53981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:05.600353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:05.685861 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-b1afca38-2742-4e1b-a159-1dcb8bf53981 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:05.687204 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b1afca38-2742-4e1b-a159-1dcb8bf53981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:05.805181 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:57:05.996537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b7601-daf1-4411-af44-0c03bf2b2366 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:05.997976 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d07b7601-daf1-4411-af44-0c03bf2b2366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:06.148088 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:06.189829 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b7601-daf1-4411-af44-0c03bf2b2366 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:06.357645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7cc27-0993-4cb0-b1e9-4e2a1d820314 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:06.358270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7cc27-0993-4cb0-b1e9-4e2a1d820314 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:06.358550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7cc27-0993-4cb0-b1e9-4e2a1d820314 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:06.358828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7cc27-0993-4cb0-b1e9-4e2a1d820314 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:06.360622 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-aee7cc27-0993-4cb0-b1e9-4e2a1d820314 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:06.465260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:06.480229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7cc27-0993-4cb0-b1e9-4e2a1d820314 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:06.481930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7cc27-0993-4cb0-b1e9-4e2a1d820314 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:06.504161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 720/1425] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:07.410542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69dbb6-a81f-4eaa-9f85-ca94fa61b1a5 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:07.411086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69dbb6-a81f-4eaa-9f85-ca94fa61b1a5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:07.411086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69dbb6-a81f-4eaa-9f85-ca94fa61b1a5 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:07.412637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:07.418547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01efbd-2d1c-47af-8314-b4a58620dd83 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:07.419033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01efbd-2d1c-47af-8314-b4a58620dd83 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:07.419134 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01efbd-2d1c-47af-8314-b4a58620dd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:07.501061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-3a01efbd-2d1c-47af-8314-b4a58620dd83 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:07.502157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3a01efbd-2d1c-47af-8314-b4a58620dd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:07.606521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:07.764096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69dbb6-a81f-4eaa-9f85-ca94fa61b1a5 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:07.765528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9d69dbb6-a81f-4eaa-9f85-ca94fa61b1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:07.856763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:07.926315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69dbb6-a81f-4eaa-9f85-ca94fa61b1a5 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:08.056982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f916e7-2b30-4177-b791-fff849955e60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:08.057600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f916e7-2b30-4177-b791-fff849955e60 service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:08.058458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f916e7-2b30-4177-b791-fff849955e60 service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:08.058862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f916e7-2b30-4177-b791-fff849955e60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:08.060912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-22f916e7-2b30-4177-b791-fff849955e60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:08.124899 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:08.169324 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f916e7-2b30-4177-b791-fff849955e60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:08.170923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f916e7-2b30-4177-b791-fff849955e60 service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:08.190413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 707/1428] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:09.221511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ebf33-68e8-4766-929a-d44be4cb50a5 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:09.222068 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ebf33-68e8-4766-929a-d44be4cb50a5 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:09.222191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ebf33-68e8-4766-929a-d44be4cb50a5 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:09.223678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:09.228812 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ca5c9-2b5b-4a41-9f6c-c877825e0cf0 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:09.229313 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ca5c9-2b5b-4a41-9f6c-c877825e0cf0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:09.229431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ca5c9-2b5b-4a41-9f6c-c877825e0cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:09.322902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-214ca5c9-2b5b-4a41-9f6c-c877825e0cf0 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:09.324192 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-214ca5c9-2b5b-4a41-9f6c-c877825e0cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:09.442003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:09.596651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ebf33-68e8-4766-929a-d44be4cb50a5 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:09.598503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9c3ebf33-68e8-4766-929a-d44be4cb50a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:09.659207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:09.783166 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ebf33-68e8-4766-929a-d44be4cb50a5 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:09.927421 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0ba7a-74db-441d-81f4-784574eb4d91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:09.927985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ba7a-74db-441d-81f4-784574eb4d91 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:09.928266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ba7a-74db-441d-81f4-784574eb4d91 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:09.928539 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ba7a-74db-441d-81f4-784574eb4d91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:09.930262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ffb0ba7a-74db-441d-81f4-784574eb4d91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:09.939052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:10.034362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0ba7a-74db-441d-81f4-784574eb4d91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:10.036051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0ba7a-74db-441d-81f4-784574eb4d91 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:10.058094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 723/1431] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:11.122623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b39efa-938f-4565-b807-d7ed2c45555c None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:11.123259 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b39efa-938f-4565-b807-d7ed2c45555c None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:11.123453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b39efa-938f-4565-b807-d7ed2c45555c None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:11.125359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:57:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:11.131955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d335b7d-c19f-4ce1-b432-228f4703cb10 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:11.132803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d335b7d-c19f-4ce1-b432-228f4703cb10 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:11.132993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d335b7d-c19f-4ce1-b432-228f4703cb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:11.242311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7d335b7d-c19f-4ce1-b432-228f4703cb10 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:11.243648 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7d335b7d-c19f-4ce1-b432-228f4703cb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:11.353592 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:11.393204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:11.564094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b39efa-938f-4565-b807-d7ed2c45555c None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:11.566061 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c0b39efa-938f-4565-b807-d7ed2c45555c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:11.594496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:11.837508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b39efa-938f-4565-b807-d7ed2c45555c None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:11.964398 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:12.000226 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149876e-49d4-498d-8b86-839bb8d1b1e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:12.001528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149876e-49d4-498d-8b86-839bb8d1b1e1 service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:12.002018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149876e-49d4-498d-8b86-839bb8d1b1e1 service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:12.002932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149876e-49d4-498d-8b86-839bb8d1b1e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:12.006727 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f149876e-49d4-498d-8b86-839bb8d1b1e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:12.186375 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149876e-49d4-498d-8b86-839bb8d1b1e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:12.189527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149876e-49d4-498d-8b86-839bb8d1b1e1 service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:12.218348 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 710/1434] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:13.137254 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de43bc-406d-4843-908d-fd9bd48d5fa8 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:13.137844 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de43bc-406d-4843-908d-fd9bd48d5fa8 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:13.137844 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de43bc-406d-4843-908d-fd9bd48d5fa8 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:13.139402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:13.144865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e39fd0-c679-44c2-8c9b-7216b25c584b None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:13.145157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e39fd0-c679-44c2-8c9b-7216b25c584b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:13.145436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e39fd0-c679-44c2-8c9b-7216b25c584b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:13.251525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-31e39fd0-c679-44c2-8c9b-7216b25c584b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:13.252782 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-31e39fd0-c679-44c2-8c9b-7216b25c584b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:13.312287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:13.376683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:13.535625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de43bc-406d-4843-908d-fd9bd48d5fa8 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:13.537147 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-14de43bc-406d-4843-908d-fd9bd48d5fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:13.707802 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de43bc-406d-4843-908d-fd9bd48d5fa8 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:13.770949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:13.842618 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fff6d7-acca-4ea3-a9de-656cc1005bb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:13.843230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fff6d7-acca-4ea3-a9de-656cc1005bb1 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:13.843535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fff6d7-acca-4ea3-a9de-656cc1005bb1 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:13.843815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fff6d7-acca-4ea3-a9de-656cc1005bb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:13.845529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-72fff6d7-acca-4ea3-a9de-656cc1005bb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:13.946223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fff6d7-acca-4ea3-a9de-656cc1005bb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:13.947945 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fff6d7-acca-4ea3-a9de-656cc1005bb1 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:13.970066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 726/1437] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:14.914788 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b581c-aa72-44c8-9ff6-4324e427c0ed None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:14.915339 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b581c-aa72-44c8-9ff6-4324e427c0ed None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:14.915339 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b581c-aa72-44c8-9ff6-4324e427c0ed None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:14.916742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 14:57:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:14.923165 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5a867-62ad-420b-8dae-bbaf2c9dad08 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:14.924030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5a867-62ad-420b-8dae-bbaf2c9dad08 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:14.924236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5a867-62ad-420b-8dae-bbaf2c9dad08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:15.014336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-50d5a867-62ad-420b-8dae-bbaf2c9dad08 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:15.015902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-50d5a867-62ad-420b-8dae-bbaf2c9dad08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:15.018455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:15.125099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 14:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:15.284262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b581c-aa72-44c8-9ff6-4324e427c0ed None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:15.286724 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-988b581c-aa72-44c8-9ff6-4324e427c0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:15.457886 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b581c-aa72-44c8-9ff6-4324e427c0ed None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:15.490859 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:15.598119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8769241-788a-424e-a42e-0463c0db9849 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:15.598756 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8769241-788a-424e-a42e-0463c0db9849 service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:15.599032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8769241-788a-424e-a42e-0463c0db9849 service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:15.599304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8769241-788a-424e-a42e-0463c0db9849 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:15.601094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c8769241-788a-424e-a42e-0463c0db9849 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:15.733881 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8769241-788a-424e-a42e-0463c0db9849 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:15.736013 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8769241-788a-424e-a42e-0463c0db9849 service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:15.758790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 713/1440] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:17.800133 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035eb5f-8b74-4cd2-a671-a5be174720a0 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:17.800795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035eb5f-8b74-4cd2-a671-a5be174720a0 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:17.800795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035eb5f-8b74-4cd2-a671-a5be174720a0 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:17.802049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:17.808604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c72a5-2647-43c6-922a-4f35fa752d98 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:17.812935 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c72a5-2647-43c6-922a-4f35fa752d98 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:17.812935 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c72a5-2647-43c6-922a-4f35fa752d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:17.854009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:17.884977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-055c72a5-2647-43c6-922a-4f35fa752d98 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:17.886464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-055c72a5-2647-43c6-922a-4f35fa752d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:17.994327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:18.035695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035eb5f-8b74-4cd2-a671-a5be174720a0 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:18.037263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-8035eb5f-8b74-4cd2-a671-a5be174720a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:18.148732 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035eb5f-8b74-4cd2-a671-a5be174720a0 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:18.183306 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:18.229140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e05487-5301-4a51-ae04-71ed7774aaf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:18.229767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e05487-5301-4a51-ae04-71ed7774aaf6 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:18.230043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e05487-5301-4a51-ae04-71ed7774aaf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:18.230316 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e05487-5301-4a51-ae04-71ed7774aaf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:18.232202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-46e05487-5301-4a51-ae04-71ed7774aaf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:18.357983 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e05487-5301-4a51-ae04-71ed7774aaf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:18.359431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e05487-5301-4a51-ae04-71ed7774aaf6 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:18.380095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 729/1443] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:20.639572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8f8d4-5ddf-441e-b249-8c4ce59ad61c None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:20.640146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8f8d4-5ddf-441e-b249-8c4ce59ad61c None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:20.640146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8f8d4-5ddf-441e-b249-8c4ce59ad61c None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:20.641509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:20.647987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cd3f8-b420-4225-b1fa-573bb8287901 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:20.648841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cd3f8-b420-4225-b1fa-573bb8287901 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:20.648841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cd3f8-b420-4225-b1fa-573bb8287901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:20.726656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-d66cd3f8-b420-4225-b1fa-573bb8287901 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:20.728016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d66cd3f8-b420-4225-b1fa-573bb8287901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:20.743196 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:20.839305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:57:20.878747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8f8d4-5ddf-441e-b249-8c4ce59ad61c None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:20.880366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b4c8f8d4-5ddf-441e-b249-8c4ce59ad61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:21.000344 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8f8d4-5ddf-441e-b249-8c4ce59ad61c None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:21.016965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:21.295235 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbbb95-4214-4fcb-ba2a-4e79174ff2ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:21.296067 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbbb95-4214-4fcb-ba2a-4e79174ff2ea service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:21.296345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbbb95-4214-4fcb-ba2a-4e79174ff2ea service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:21.296620 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbbb95-4214-4fcb-ba2a-4e79174ff2ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:21.298402 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-bebbbb95-4214-4fcb-ba2a-4e79174ff2ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:21.443594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbbb95-4214-4fcb-ba2a-4e79174ff2ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:21.447248 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbbb95-4214-4fcb-ba2a-4e79174ff2ea service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:21.467487 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 716/1446] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:24.164716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb26b41-509b-4ac7-8da3-25bdbff7689a None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:24.165314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb26b41-509b-4ac7-8da3-25bdbff7689a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:24.165314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb26b41-509b-4ac7-8da3-25bdbff7689a None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:24.166896 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:24.173618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22718ad3-ecff-47d4-9f3e-20ea07dc36f9 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:24.174199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22718ad3-ecff-47d4-9f3e-20ea07dc36f9 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:24.174199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22718ad3-ecff-47d4-9f3e-20ea07dc36f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:24.228969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:24.250319 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-22718ad3-ecff-47d4-9f3e-20ea07dc36f9 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:24.251797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-22718ad3-ecff-47d4-9f3e-20ea07dc36f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:24.365644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:57:24.411580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb26b41-509b-4ac7-8da3-25bdbff7689a None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:24.414581 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-bbb26b41-509b-4ac7-8da3-25bdbff7689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:24.548327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb26b41-509b-4ac7-8da3-25bdbff7689a None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:24.603145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:24.650244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39f1bf-b29a-4e3b-aacf-0b922bcef705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:24.650864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39f1bf-b29a-4e3b-aacf-0b922bcef705 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:24.651139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39f1bf-b29a-4e3b-aacf-0b922bcef705 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:24.651407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39f1bf-b29a-4e3b-aacf-0b922bcef705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:24.654654 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9e39f1bf-b29a-4e3b-aacf-0b922bcef705 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:24.783789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39f1bf-b29a-4e3b-aacf-0b922bcef705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:24.785396 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39f1bf-b29a-4e3b-aacf-0b922bcef705 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:24.805608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 732/1449] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:57:27.069573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd65c4-2b2f-4c11-8be7-baa802f813c3 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:27.070401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd65c4-2b2f-4c11-8be7-baa802f813c3 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:27.070401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd65c4-2b2f-4c11-8be7-baa802f813c3 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:27.071498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:27.078083 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811010e1-0ee9-4cbe-b4df-dcd063753915 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:27.078654 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811010e1-0ee9-4cbe-b4df-dcd063753915 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:27.078654 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811010e1-0ee9-4cbe-b4df-dcd063753915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:27.151968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-811010e1-0ee9-4cbe-b4df-dcd063753915 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:27.153452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-811010e1-0ee9-4cbe-b4df-dcd063753915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:27.177423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:27.274544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:27.316159 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd65c4-2b2f-4c11-8be7-baa802f813c3 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:27.317867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-45dd65c4-2b2f-4c11-8be7-baa802f813c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:27.441551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd65c4-2b2f-4c11-8be7-baa802f813c3 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:27.464001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:27.528242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8887edf-1e2a-499d-8432-90489b79a09b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:27.528962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8887edf-1e2a-499d-8432-90489b79a09b service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:27.529239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8887edf-1e2a-499d-8432-90489b79a09b service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:27.529641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8887edf-1e2a-499d-8432-90489b79a09b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:27.531213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a8887edf-1e2a-499d-8432-90489b79a09b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:27.655154 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8887edf-1e2a-499d-8432-90489b79a09b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:27.656682 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8887edf-1e2a-499d-8432-90489b79a09b service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:27.676109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 719/1452] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:29.923299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e876dc-3df2-4eb6-a2d2-541771a0ccbd None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:29.923892 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e876dc-3df2-4eb6-a2d2-541771a0ccbd None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:29.923892 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e876dc-3df2-4eb6-a2d2-541771a0ccbd None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:29.925262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:29.931176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff6835-7ba3-4674-bd0d-816c57963bc6 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:29.931659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff6835-7ba3-4674-bd0d-816c57963bc6 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:29.931756 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff6835-7ba3-4674-bd0d-816c57963bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:30.000284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:30.004330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-80ff6835-7ba3-4674-bd0d-816c57963bc6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:30.006100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-80ff6835-7ba3-4674-bd0d-816c57963bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:30.111396 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:30.151978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e876dc-3df2-4eb6-a2d2-541771a0ccbd None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:30.153796 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c8e876dc-3df2-4eb6-a2d2-541771a0ccbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:30.267000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e876dc-3df2-4eb6-a2d2-541771a0ccbd None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:30.317465 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:30.345599 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971a8d2-c1e9-4907-8add-d7aba0332575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:30.346221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971a8d2-c1e9-4907-8add-d7aba0332575 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:30.346496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971a8d2-c1e9-4907-8add-d7aba0332575 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:30.346760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971a8d2-c1e9-4907-8add-d7aba0332575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:30.348574 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1971a8d2-c1e9-4907-8add-d7aba0332575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:30.485887 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971a8d2-c1e9-4907-8add-d7aba0332575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:30.487360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971a8d2-c1e9-4907-8add-d7aba0332575 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:30.508880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 735/1455] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:32.697057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affc47e-be41-48db-8374-d4dfd422263e None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:32.698006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affc47e-be41-48db-8374-d4dfd422263e None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:32.698006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affc47e-be41-48db-8374-d4dfd422263e None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:32.699071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:32.706068 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02de5b-aee6-44c8-9390-5d865085fc50 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:32.706932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02de5b-aee6-44c8-9390-5d865085fc50 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:32.706932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02de5b-aee6-44c8-9390-5d865085fc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:32.789865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7e02de5b-aee6-44c8-9390-5d865085fc50 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:32.791009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7e02de5b-aee6-44c8-9390-5d865085fc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:32.825776 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:32.909067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:32.949794 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affc47e-be41-48db-8374-d4dfd422263e None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:32.952221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-2affc47e-be41-48db-8374-d4dfd422263e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:33.082871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affc47e-be41-48db-8374-d4dfd422263e None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:33.123479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:33.196382 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b3748-0ec5-4963-a58b-f084e76b92b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:33.197025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b3748-0ec5-4963-a58b-f084e76b92b9 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:33.197310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b3748-0ec5-4963-a58b-f084e76b92b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:33.197594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b3748-0ec5-4963-a58b-f084e76b92b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:33.199297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c71b3748-0ec5-4963-a58b-f084e76b92b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:33.336059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b3748-0ec5-4963-a58b-f084e76b92b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:33.337646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b3748-0ec5-4963-a58b-f084e76b92b9 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:33.360289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 722/1458] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:35.651753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9386b7-b66b-47a4-aec5-dab349b5e575 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:35.652367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9386b7-b66b-47a4-aec5-dab349b5e575 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:35.652367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9386b7-b66b-47a4-aec5-dab349b5e575 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:35.653759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:35.660531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ef65f-8399-4c5d-8a96-7648c1570c2a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:35.661293 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ef65f-8399-4c5d-8a96-7648c1570c2a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:35.661293 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ef65f-8399-4c5d-8a96-7648c1570c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:35.750589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-635ef65f-8399-4c5d-8a96-7648c1570c2a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:35.751936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-635ef65f-8399-4c5d-8a96-7648c1570c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:35.762652 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:35.877544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:35.918797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9386b7-b66b-47a4-aec5-dab349b5e575 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:35.920586 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ce9386b7-b66b-47a4-aec5-dab349b5e575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:36.055536 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9386b7-b66b-47a4-aec5-dab349b5e575 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:36.126606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:36.152142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46176db-d3d9-4043-ae9d-74399cdddd20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:36.152684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46176db-d3d9-4043-ae9d-74399cdddd20 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:36.152965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46176db-d3d9-4043-ae9d-74399cdddd20 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:36.153241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46176db-d3d9-4043-ae9d-74399cdddd20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:36.155525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d46176db-d3d9-4043-ae9d-74399cdddd20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:36.328701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46176db-d3d9-4043-ae9d-74399cdddd20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:36.331500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46176db-d3d9-4043-ae9d-74399cdddd20 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:36.358054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 738/1461] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:57:38.653221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffbd54-72f4-41a9-9d54-566182107dba None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:38.653890 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffbd54-72f4-41a9-9d54-566182107dba None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:38.654237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffbd54-72f4-41a9-9d54-566182107dba None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:38.656990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:57:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:38.664088 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131744cb-4f3b-4042-9b2b-fc82ec20cf17 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:38.664664 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131744cb-4f3b-4042-9b2b-fc82ec20cf17 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:38.664664 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131744cb-4f3b-4042-9b2b-fc82ec20cf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:38.739094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-131744cb-4f3b-4042-9b2b-fc82ec20cf17 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:38.740720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-131744cb-4f3b-4042-9b2b-fc82ec20cf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:38.788206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:38.860936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:38.905309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffbd54-72f4-41a9-9d54-566182107dba None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:38.906846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b4ffbd54-72f4-41a9-9d54-566182107dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:39.046945 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffbd54-72f4-41a9-9d54-566182107dba None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:39.098840 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:39.145497 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8777b-283d-4ec2-bfcb-cba4b18881c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:39.146131 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8777b-283d-4ec2-bfcb-cba4b18881c0 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:39.146410 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8777b-283d-4ec2-bfcb-cba4b18881c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:39.146685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8777b-283d-4ec2-bfcb-cba4b18881c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:39.148488 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ded8777b-283d-4ec2-bfcb-cba4b18881c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:39.300117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded8777b-283d-4ec2-bfcb-cba4b18881c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:39.301622 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded8777b-283d-4ec2-bfcb-cba4b18881c0 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:39.325831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 725/1464] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:41.523879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1649b2-6acc-4ed7-8a00-20bd726f039c None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:41.524547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1649b2-6acc-4ed7-8a00-20bd726f039c None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:41.524547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1649b2-6acc-4ed7-8a00-20bd726f039c None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:41.526188 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:57:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:41.532846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da0163-fe32-4c7a-9b20-269ded601a54 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:41.533429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da0163-fe32-4c7a-9b20-269ded601a54 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:41.533429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da0163-fe32-4c7a-9b20-269ded601a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:41.610453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-f8da0163-fe32-4c7a-9b20-269ded601a54 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:41.611765 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f8da0163-fe32-4c7a-9b20-269ded601a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:41.631643 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:41.725241 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:41.765480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1649b2-6acc-4ed7-8a00-20bd726f039c None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:41.767069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2a1649b2-6acc-4ed7-8a00-20bd726f039c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:41.946500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1649b2-6acc-4ed7-8a00-20bd726f039c None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:42.046656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:42.060903 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c7651-a2cc-467b-b239-232580b097b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:42.062188 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c7651-a2cc-467b-b239-232580b097b4 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:42.062697 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c7651-a2cc-467b-b239-232580b097b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:42.063199 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c7651-a2cc-467b-b239-232580b097b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:42.066611 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-336c7651-a2cc-467b-b239-232580b097b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:42.221992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c7651-a2cc-467b-b239-232580b097b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:42.223450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c7651-a2cc-467b-b239-232580b097b4 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:42.246161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 741/1467] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:44.456350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4976f-c6d8-48bb-8cf7-b7cbc3b4fe53 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:44.456901 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4976f-c6d8-48bb-8cf7-b7cbc3b4fe53 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:44.456901 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4976f-c6d8-48bb-8cf7-b7cbc3b4fe53 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:44.458470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 14:57:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:57:44.465792 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde39172-d42a-4330-a2a1-6c25422252db None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:44.466625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde39172-d42a-4330-a2a1-6c25422252db None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:44.469921 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde39172-d42a-4330-a2a1-6c25422252db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:44.548408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-cde39172-d42a-4330-a2a1-6c25422252db None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:57:44.549646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-cde39172-d42a-4330-a2a1-6c25422252db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:44.617508 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:44.712881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 14:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:57:44.752507 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4976f-c6d8-48bb-8cf7-b7cbc3b4fe53 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:57:44.754058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-66d4976f-c6d8-48bb-8cf7-b7cbc3b4fe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:44.875623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4976f-c6d8-48bb-8cf7-b7cbc3b4fe53 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:57:44.930031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:44.959476 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb41c34-4241-4ac2-9ec7-c1e5574112a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:57:44.960192 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb41c34-4241-4ac2-9ec7-c1e5574112a8 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:57:44.960522 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb41c34-4241-4ac2-9ec7-c1e5574112a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:57:44.960858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb41c34-4241-4ac2-9ec7-c1e5574112a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:57:44.962604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-7fb41c34-4241-4ac2-9ec7-c1e5574112a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:57:45.096767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb41c34-4241-4ac2-9ec7-c1e5574112a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:57:45.098349 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb41c34-4241-4ac2-9ec7-c1e5574112a8 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:57:45.120394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 728/1470] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:57:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:57:54.709080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=22 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:57:55.117908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=66 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:06.393826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15c726-1ebd-44d7-9736-65bddbebc184 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:06.394672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15c726-1ebd-44d7-9736-65bddbebc184 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:06.394672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15c726-1ebd-44d7-9736-65bddbebc184 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:06.396721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 743/1471] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:58:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:58:06.403896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08522d-faf7-4972-b078-51af7f53dc6d None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:06.404535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08522d-faf7-4972-b078-51af7f53dc6d None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:06.404535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08522d-faf7-4972-b078-51af7f53dc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:06.496021 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-5a08522d-faf7-4972-b078-51af7f53dc6d None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:58:06.497485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-5a08522d-faf7-4972-b078-51af7f53dc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:06.647988 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 729/1472] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:58:06.749941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99218c44-b5f8-4b28-be73-59cf93880b2e None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:06.750157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99218c44-b5f8-4b28-be73-59cf93880b2e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:06.750452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99218c44-b5f8-4b28-be73-59cf93880b2e None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:06.753463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 744/1473] 10.222.0.94 () {54 vars in 966 bytes} [Tue Jul 26 14:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:58:06.757733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873e078-5929-49b8-a2d0-deb3b4177682 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:06.758022 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873e078-5929-49b8-a2d0-deb3b4177682 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:06.758301 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873e078-5929-49b8-a2d0-deb3b4177682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:06.855336 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-9873e078-5929-49b8-a2d0-deb3b4177682 None None] MFA Rules not processed for user `b153935910c142aa80792e111872318a`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:58:06.858021 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9873e078-5929-49b8-a2d0-deb3b4177682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:06.899922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:06.995029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 730/1474] 10.222.0.94 () {58 vars in 1063 bytes} [Tue Jul 26 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:58:07.004244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99218c44-b5f8-4b28-be73-59cf93880b2e None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:07.005756 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-99218c44-b5f8-4b28-be73-59cf93880b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:07.135908 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99218c44-b5f8-4b28-be73-59cf93880b2e None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:07.235662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ed421-ed5f-4822-902e-c04f30137e7a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b153935910c142aa80792e111872318a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:07.236293 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ed421-ed5f-4822-902e-c04f30137e7a service glance] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:07.236569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ed421-ed5f-4822-902e-c04f30137e7a service glance] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:07.236842 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ed421-ed5f-4822-902e-c04f30137e7a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:07.238547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4d4ed421-ed5f-4822-902e-c04f30137e7a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:07.326192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:07.409185 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ed421-ed5f-4822-902e-c04f30137e7a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:07.411120 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ed421-ed5f-4822-902e-c04f30137e7a service glance] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:07.435502 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 745/1475] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:58:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:58:10.426625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30dafe-1ff2-4696-90c0-3f0b0b15276f None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:10.427542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30dafe-1ff2-4696-90c0-3f0b0b15276f None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:10.427542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30dafe-1ff2-4696-90c0-3f0b0b15276f None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:10.429685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 731/1476] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:58:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:58:10.438871 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fb8aa-de45-41b8-bfcc-77ffe5343c7b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:10.439810 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fb8aa-de45-41b8-bfcc-77ffe5343c7b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:10.440130 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fb8aa-de45-41b8-bfcc-77ffe5343c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:10.595773 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-871fb8aa-de45-41b8-bfcc-77ffe5343c7b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:58:10.598335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-871fb8aa-de45-41b8-bfcc-77ffe5343c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:10.841045 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 746/1477] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:58:10.926194 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30dafe-1ff2-4696-90c0-3f0b0b15276f None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:10.928017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3f30dafe-1ff2-4696-90c0-3f0b0b15276f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:11.155928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30dafe-1ff2-4696-90c0-3f0b0b15276f None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:11.240122 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:11.321375 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca9ec0-f97d-4f20-abb2-5886dbb7d4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:11.322347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca9ec0-f97d-4f20-abb2-5886dbb7d4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:11.322647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca9ec0-f97d-4f20-abb2-5886dbb7d4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:11.322967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca9ec0-f97d-4f20-abb2-5886dbb7d4b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:11.323831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca9ec0-f97d-4f20-abb2-5886dbb7d4b4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:11.326174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: WARNING py.warnings [None req-1aca9ec0-f97d-4f20-abb2-5886dbb7d4b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 14:58:11.326174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: warnings.warn(msg) Jul 26 14:58:11.326174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: Jul 26 14:58:11.327489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca9ec0-f97d-4f20-abb2-5886dbb7d4b4 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:11.366946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 732/1478] 10.222.0.94 () {56 vars in 1108 bytes} [Tue Jul 26 14:58:10 2022] GET /identity/v3/projects => generated 2553 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 14:58:14.197077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecff4b6-890a-4723-99da-b115c3a3113d None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:14.197860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecff4b6-890a-4723-99da-b115c3a3113d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:14.198065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecff4b6-890a-4723-99da-b115c3a3113d None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:14.201391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 747/1479] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:58:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:58:14.210695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66de53-35fc-479c-9c81-2030b794fe0e None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:14.211690 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66de53-35fc-479c-9c81-2030b794fe0e None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:14.211894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66de53-35fc-479c-9c81-2030b794fe0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:14.320913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ee66de53-35fc-479c-9c81-2030b794fe0e None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:58:14.322691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ee66de53-35fc-479c-9c81-2030b794fe0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:14.471936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 733/1480] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:58:14.519848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecff4b6-890a-4723-99da-b115c3a3113d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:14.521511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2ecff4b6-890a-4723-99da-b115c3a3113d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:14.533470 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:14.738177 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecff4b6-890a-4723-99da-b115c3a3113d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:14.848558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:14.893885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fa224-f25d-4c09-9db7-42fb663a53c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:14.894514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fa224-f25d-4c09-9db7-42fb663a53c3 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:14.894795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fa224-f25d-4c09-9db7-42fb663a53c3 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:14.895062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fa224-f25d-4c09-9db7-42fb663a53c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:14.896890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e94fa224-f25d-4c09-9db7-42fb663a53c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:15.037647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fa224-f25d-4c09-9db7-42fb663a53c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:15.039353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fa224-f25d-4c09-9db7-42fb663a53c3 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:15.064399 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 748/1481] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:58:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:58:17.732475 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4803224-ecef-4512-acef-60d7040f1232 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:17.733325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4803224-ecef-4512-acef-60d7040f1232 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:17.733495 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4803224-ecef-4512-acef-60d7040f1232 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:17.736187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 734/1482] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:58:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:58:17.745072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ca018-a150-4ff5-bae0-87ed52cff420 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:17.748319 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ca018-a150-4ff5-bae0-87ed52cff420 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:17.748319 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ca018-a150-4ff5-bae0-87ed52cff420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:17.866145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-de3ca018-a150-4ff5-bae0-87ed52cff420 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:58:17.867730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-de3ca018-a150-4ff5-bae0-87ed52cff420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:17.938500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:18.046495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 749/1483] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:58:18.090032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4803224-ecef-4512-acef-60d7040f1232 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:18.093138 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d4803224-ecef-4512-acef-60d7040f1232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:18.143895 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:18.276814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4803224-ecef-4512-acef-60d7040f1232 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:18.426943 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a9d82-e8d9-4f1e-9aab-45a80e0da2da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:18.428315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a9d82-e8d9-4f1e-9aab-45a80e0da2da service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:18.428831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a9d82-e8d9-4f1e-9aab-45a80e0da2da service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:18.429304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a9d82-e8d9-4f1e-9aab-45a80e0da2da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:18.432478 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ab8a9d82-e8d9-4f1e-9aab-45a80e0da2da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:18.589433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:18.649397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a9d82-e8d9-4f1e-9aab-45a80e0da2da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:18.651452 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a9d82-e8d9-4f1e-9aab-45a80e0da2da service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:18.691356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 735/1484] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:58:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:58:21.448249 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12f55c-d3a9-4b08-b8cf-dffcdf4b5270 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:21.449640 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12f55c-d3a9-4b08-b8cf-dffcdf4b5270 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:21.449782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12f55c-d3a9-4b08-b8cf-dffcdf4b5270 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:21.452631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 750/1485] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:58:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:58:21.459843 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e6502-65ea-42f5-8d82-d821c96b2414 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:21.460663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e6502-65ea-42f5-8d82-d821c96b2414 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:21.460663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e6502-65ea-42f5-8d82-d821c96b2414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:21.573299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-3b3e6502-65ea-42f5-8d82-d821c96b2414 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:58:21.575207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3b3e6502-65ea-42f5-8d82-d821c96b2414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:21.777966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 736/1486] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 14:58:21.894141 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12f55c-d3a9-4b08-b8cf-dffcdf4b5270 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:21.901779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9e12f55c-d3a9-4b08-b8cf-dffcdf4b5270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:22.042332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12f55c-d3a9-4b08-b8cf-dffcdf4b5270 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:22.145403 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:22.147925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8fd462-8940-452a-9079-fefd0df5acf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:22.148680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fd462-8940-452a-9079-fefd0df5acf8 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:22.149061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fd462-8940-452a-9079-fefd0df5acf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:22.149380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fd462-8940-452a-9079-fefd0df5acf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:22.151297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2c8fd462-8940-452a-9079-fefd0df5acf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:22.324271 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8fd462-8940-452a-9079-fefd0df5acf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:22.326556 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8fd462-8940-452a-9079-fefd0df5acf8 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:22.353465 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 751/1487] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:58:25.576903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b201d-3511-4eb6-85c5-f9f26132865e None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:25.577546 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b201d-3511-4eb6-85c5-f9f26132865e None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:25.577546 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b201d-3511-4eb6-85c5-f9f26132865e None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:25.579075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 737/1488] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:58:25.585600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a042927-c346-48d7-9876-6209347cbc9e None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:25.586385 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a042927-c346-48d7-9876-6209347cbc9e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:25.586385 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a042927-c346-48d7-9876-6209347cbc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:25.690176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-9a042927-c346-48d7-9876-6209347cbc9e None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:58:25.691712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9a042927-c346-48d7-9876-6209347cbc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:25.777571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:25.876154 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 752/1489] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:58:25.930353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b201d-3511-4eb6-85c5-f9f26132865e None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:25.932020 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f76b201d-3511-4eb6-85c5-f9f26132865e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:26.000575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:26.132818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b201d-3511-4eb6-85c5-f9f26132865e None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:26.303956 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf377e-244c-450e-a2df-aeb4ff87a9d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:26.305033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf377e-244c-450e-a2df-aeb4ff87a9d4 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:26.305534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf377e-244c-450e-a2df-aeb4ff87a9d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:26.306025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf377e-244c-450e-a2df-aeb4ff87a9d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:26.310078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-19cf377e-244c-450e-a2df-aeb4ff87a9d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:26.470201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:26.537198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf377e-244c-450e-a2df-aeb4ff87a9d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:26.539116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf377e-244c-450e-a2df-aeb4ff87a9d4 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:26.570116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 738/1490] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:58:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 14:58:29.634001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e914392-4e4d-419d-91b0-bba20c1765cf None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:29.634606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e914392-4e4d-419d-91b0-bba20c1765cf None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:29.634782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e914392-4e4d-419d-91b0-bba20c1765cf None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:29.637543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 753/1491] 10.222.0.94 () {54 vars in 926 bytes} [Tue Jul 26 14:58:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 14:58:29.646181 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71ad94-315a-4c77-9bdf-0ff495cf5b22 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:29.646789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71ad94-315a-4c77-9bdf-0ff495cf5b22 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:29.646967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71ad94-315a-4c77-9bdf-0ff495cf5b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:29.754758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-0b71ad94-315a-4c77-9bdf-0ff495cf5b22 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 14:58:29.756206 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0b71ad94-315a-4c77-9bdf-0ff495cf5b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:29.905960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 739/1492] 10.222.0.94 () {58 vars in 1023 bytes} [Tue Jul 26 14:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 14:58:29.950735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e914392-4e4d-419d-91b0-bba20c1765cf None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 14:58:29.952531 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2e914392-4e4d-419d-91b0-bba20c1765cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:30.116107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e914392-4e4d-419d-91b0-bba20c1765cf None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 14:58:30.212783 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8f7a6-54e2-4f33-8377-ebc7dd9cd1a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 14:58:30.213554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8f7a6-54e2-4f33-8377-ebc7dd9cd1a6 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 14:58:30.214617 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8f7a6-54e2-4f33-8377-ebc7dd9cd1a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 14:58:30.215623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8f7a6-54e2-4f33-8377-ebc7dd9cd1a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 14:58:30.217914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f2c8f7a6-54e2-4f33-8377-ebc7dd9cd1a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 14:58:30.229859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:30.414980 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c8f7a6-54e2-4f33-8377-ebc7dd9cd1a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 14:58:30.416704 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c8f7a6-54e2-4f33-8377-ebc7dd9cd1a6 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 14:58:30.467659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 754/1493] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 14:58:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 14:58:39.903602 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=82 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 14:58:40.451302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=50 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:00:29.666133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6a693-140c-4072-809c-b798da6024cf None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:29.666777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6a693-140c-4072-809c-b798da6024cf None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:29.666777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6a693-140c-4072-809c-b798da6024cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:29.779000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-afb6a693-140c-4072-809c-b798da6024cf None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:00:29.781080 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-afb6a693-140c-4072-809c-b798da6024cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:29.920345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 740/1494] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:00:29.929388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4ea25-3d82-4d07-b599-f8064f462793 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:29.929968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4ea25-3d82-4d07-b599-f8064f462793 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:29.929968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4ea25-3d82-4d07-b599-f8064f462793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:29.953499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-5fe4ea25-3d82-4d07-b599-f8064f462793 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:00:29.954750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5fe4ea25-3d82-4d07-b599-f8064f462793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:29.965323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 755/1495] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:00:29.970376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6a693-140c-4072-809c-b798da6024cf None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:29.981910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6a693-140c-4072-809c-b798da6024cf None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:29.987289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694878ba-7812-4794-9183-e10afbed6133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:29.988038 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694878ba-7812-4794-9183-e10afbed6133 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:29.988388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694878ba-7812-4794-9183-e10afbed6133 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:29.988714 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694878ba-7812-4794-9183-e10afbed6133 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:29.989501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694878ba-7812-4794-9183-e10afbed6133 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:00:29.989853 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694878ba-7812-4794-9183-e10afbed6133 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:29.991184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: WARNING py.warnings [None req-694878ba-7812-4794-9183-e10afbed6133 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:00:29.991184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: warnings.warn(msg) Jul 26 15:00:29.991184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: Jul 26 15:00:29.992340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694878ba-7812-4794-9183-e10afbed6133 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:30.007559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 741/1496] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:00:30.016847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4ea25-3d82-4d07-b599-f8064f462793 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:30.028838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4ea25-3d82-4d07-b599-f8064f462793 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:30.034364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c685e3-e56e-436d-ad1b-a05b4308a4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:30.035000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c685e3-e56e-436d-ad1b-a05b4308a4e9 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:30.035279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c685e3-e56e-436d-ad1b-a05b4308a4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:30.035589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c685e3-e56e-436d-ad1b-a05b4308a4e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:30.036280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c685e3-e56e-436d-ad1b-a05b4308a4e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:30.039319 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: WARNING py.warnings [None req-20c685e3-e56e-436d-ad1b-a05b4308a4e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:00:30.039319 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: warnings.warn(msg) Jul 26 15:00:30.039319 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: Jul 26 15:00:30.040578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c685e3-e56e-436d-ad1b-a05b4308a4e9 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:30.220621 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 756/1497] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:00:30 2022] POST /identity/v3/projects => generated 379 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:00:30.225034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694878ba-7812-4794-9183-e10afbed6133 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:30.235561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694878ba-7812-4794-9183-e10afbed6133 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:30.292312 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:00:30.303567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6671610-ddec-4c41-bd71-33a4e119a93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:30.303567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671610-ddec-4c41-bd71-33a4e119a93d admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:30.303567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671610-ddec-4c41-bd71-33a4e119a93d admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:30.303567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671610-ddec-4c41-bd71-33a4e119a93d admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:30.304044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6671610-ddec-4c41-bd71-33a4e119a93d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:30.304044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: WARNING py.warnings [None req-d6671610-ddec-4c41-bd71-33a4e119a93d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:00:30.304044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: warnings.warn(msg) Jul 26 15:00:30.304044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: Jul 26 15:00:30.304376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6671610-ddec-4c41-bd71-33a4e119a93d admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:30.648786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 742/1498] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:00:30 2022] POST /identity/v3/users => generated 290 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:00:30.654099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c685e3-e56e-436d-ad1b-a05b4308a4e9 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:30.666276 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c685e3-e56e-436d-ad1b-a05b4308a4e9 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:30.671744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9259c8-a831-4afc-96c0-580f5ae01245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:30.672397 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9259c8-a831-4afc-96c0-580f5ae01245 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:30.672679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9259c8-a831-4afc-96c0-580f5ae01245 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:30.672959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9259c8-a831-4afc-96c0-580f5ae01245 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:30.673715 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9259c8-a831-4afc-96c0-580f5ae01245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:30.674762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: WARNING py.warnings [None req-3e9259c8-a831-4afc-96c0-580f5ae01245 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:00:30.674762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: warnings.warn(msg) Jul 26 15:00:30.674762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: Jul 26 15:00:30.675709 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9259c8-a831-4afc-96c0-580f5ae01245 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:30.690898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 757/1499] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:00:30.696359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6671610-ddec-4c41-bd71-33a4e119a93d admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:30.707551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6671610-ddec-4c41-bd71-33a4e119a93d admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:30.713827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74010c-7146-4db1-a243-cd2dfe0d84b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:30.714493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74010c-7146-4db1-a243-cd2dfe0d84b6 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:30.714795 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74010c-7146-4db1-a243-cd2dfe0d84b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:30.715073 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74010c-7146-4db1-a243-cd2dfe0d84b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:30.715818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74010c-7146-4db1-a243-cd2dfe0d84b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:30.716841 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: WARNING py.warnings [None req-ea74010c-7146-4db1-a243-cd2dfe0d84b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:00:30.716841 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: warnings.warn(msg) Jul 26 15:00:30.716841 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: Jul 26 15:00:30.717743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74010c-7146-4db1-a243-cd2dfe0d84b6 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:30.736889 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 743/1500] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:00:30.741565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9259c8-a831-4afc-96c0-580f5ae01245 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:30.752667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9259c8-a831-4afc-96c0-580f5ae01245 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:30.757902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54acb16-fc84-4f66-82c2-f9ca999f2cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:30.758728 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54acb16-fc84-4f66-82c2-f9ca999f2cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:30.759038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54acb16-fc84-4f66-82c2-f9ca999f2cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:30.759317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54acb16-fc84-4f66-82c2-f9ca999f2cf4 admin admin] PATH_INFO: `/v3/projects/796c5b646abb499d84b6fd9f7db1704f/users/1f7f351f490e4f9bbb3471d990db7528/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:30.778418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54acb16-fc84-4f66-82c2-f9ca999f2cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=796c5b646abb499d84b6fd9f7db1704f, user_id=1f7f351f490e4f9bbb3471d990db7528, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:30.779612 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: WARNING py.warnings [None req-a54acb16-fc84-4f66-82c2-f9ca999f2cf4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:00:30.779612 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: warnings.warn(msg) Jul 26 15:00:30.779612 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: Jul 26 15:00:30.781888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54acb16-fc84-4f66-82c2-f9ca999f2cf4 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:30.892020 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 758/1501] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:00:30 2022] PUT /identity/v3/projects/796c5b646abb499d84b6fd9f7db1704f/users/1f7f351f490e4f9bbb3471d990db7528/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:00:30.906198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de91e90-50c1-4e59-8d2a-ecbb0742a696 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:30.906701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de91e90-50c1-4e59-8d2a-ecbb0742a696 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:30.907182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de91e90-50c1-4e59-8d2a-ecbb0742a696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:30.954170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-6de91e90-50c1-4e59-8d2a-ecbb0742a696 None None] MFA Rules not processed for user `1f7f351f490e4f9bbb3471d990db7528`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:00:30.955440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-6de91e90-50c1-4e59-8d2a-ecbb0742a696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:31.041324 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 744/1502] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:00:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:00:31.055595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351dc95-ea03-4c41-a5a2-14d89168a602 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:31.056339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351dc95-ea03-4c41-a5a2-14d89168a602 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:31.056339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351dc95-ea03-4c41-a5a2-14d89168a602 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:31.057870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 759/1503] 10.222.0.94 () {54 vars in 964 bytes} [Tue Jul 26 15:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:00:31.063572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c107f-cad5-4f3d-a958-b8444f0edd47 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:31.063860 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c107f-cad5-4f3d-a958-b8444f0edd47 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:31.064148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c107f-cad5-4f3d-a958-b8444f0edd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:31.132543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-422c107f-cad5-4f3d-a958-b8444f0edd47 None None] MFA Rules not processed for user `7448959048f746a0ae1588ff839deb0c`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:00:31.133761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-422c107f-cad5-4f3d-a958-b8444f0edd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:31.219364 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 745/1504] 10.222.0.94 () {58 vars in 1061 bytes} [Tue Jul 26 15:00:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:00:31.228447 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351dc95-ea03-4c41-a5a2-14d89168a602 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:31.241016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351dc95-ea03-4c41-a5a2-14d89168a602 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:31.244779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6be719-4409-46cd-91ae-410a674d5724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:31.245433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6be719-4409-46cd-91ae-410a674d5724 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:31.245711 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6be719-4409-46cd-91ae-410a674d5724 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:31.246005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6be719-4409-46cd-91ae-410a674d5724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:31.257595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6be719-4409-46cd-91ae-410a674d5724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:31.260524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6be719-4409-46cd-91ae-410a674d5724 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:31.278270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 760/1505] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:00:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:00:31.346085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ceaee-c35c-42b2-a881-e41ae6d87b19 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:31.346085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ceaee-c35c-42b2-a881-e41ae6d87b19 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:31.346085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ceaee-c35c-42b2-a881-e41ae6d87b19 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:31.346085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 746/1506] 10.222.0.94 () {52 vars in 830 bytes} [Tue Jul 26 15:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 15:00:31.348119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6be719-4409-46cd-91ae-410a674d5724 service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:31.415140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6be719-4409-46cd-91ae-410a674d5724 service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:31.419914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:31.420544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:31.420818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:31.421084 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] PATH_INFO: `/v3/users/1f7f351f490e4f9bbb3471d990db7528` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:31.422245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f7f351f490e4f9bbb3471d990db7528)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:31.423255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: WARNING py.warnings [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:00:31.423255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: warnings.warn(msg) Jul 26 15:00:31.423255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: Jul 26 15:00:31.424656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:31.717526 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f7f351f490e4f9bbb3471d990db7528'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:31.731387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f7f351f490e4f9bbb3471d990db7528'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:31.819368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f7f351f490e4f9bbb3471d990db7528'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:31.838860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:00:31.841613 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f7f351f490e4f9bbb3471d990db7528'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:31.842389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f7f351f490e4f9bbb3471d990db7528'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:31.849042 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 761/1507] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:00:31 2022] DELETE /identity/v3/users/1f7f351f490e4f9bbb3471d990db7528 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:00:31.853852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e5419-c3bc-446c-9348-51ddd246d85d None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:31.854469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e5419-c3bc-446c-9348-51ddd246d85d None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:31.854469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e5419-c3bc-446c-9348-51ddd246d85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:31.874979 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-de3e5419-c3bc-446c-9348-51ddd246d85d None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:00:31.876259 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-de3e5419-c3bc-446c-9348-51ddd246d85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:31.888263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:00:31.941619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 747/1508] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:00:31.953232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:31.954659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:32.036957 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68d6ff-6022-42ec-8626-ef5bbed037b6 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:32.040674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cb1a1-ea21-4d9a-a37f-730e1e1ba871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:32.041301 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cb1a1-ea21-4d9a-a37f-730e1e1ba871 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:32.041579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cb1a1-ea21-4d9a-a37f-730e1e1ba871 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:32.041850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cb1a1-ea21-4d9a-a37f-730e1e1ba871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:32.051646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807cb1a1-ea21-4d9a-a37f-730e1e1ba871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:32.057554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807cb1a1-ea21-4d9a-a37f-730e1e1ba871 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:32.064890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 762/1509] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:00:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:00:32.622570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e5419-c3bc-446c-9348-51ddd246d85d None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:00:32.625105 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-de3e5419-c3bc-446c-9348-51ddd246d85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:00:32.650609 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e5419-c3bc-446c-9348-51ddd246d85d None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:00:32.658739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94ce39-2edc-4d69-a309-8e7101d79153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:00:32.659817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94ce39-2edc-4d69-a309-8e7101d79153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:00:32.660288 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94ce39-2edc-4d69-a309-8e7101d79153 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:00:32.660750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94ce39-2edc-4d69-a309-8e7101d79153 admin admin] PATH_INFO: `/v3/projects/796c5b646abb499d84b6fd9f7db1704f` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:00:32.662975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94ce39-2edc-4d69-a309-8e7101d79153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=796c5b646abb499d84b6fd9f7db1704f)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:00:32.664891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: WARNING py.warnings [None req-2c94ce39-2edc-4d69-a309-8e7101d79153 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:00:32.664891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: warnings.warn(msg) Jul 26 15:00:32.664891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: Jul 26 15:00:32.667237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94ce39-2edc-4d69-a309-8e7101d79153 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:00:32.825230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2c94ce39-2edc-4d69-a309-8e7101d79153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '796c5b646abb499d84b6fd9f7db1704f', 'request_id': 'req-2c94ce39-2edc-4d69-a309-8e7101d79153'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:32.840367 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2c94ce39-2edc-4d69-a309-8e7101d79153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '796c5b646abb499d84b6fd9f7db1704f', 'request_id': 'req-2c94ce39-2edc-4d69-a309-8e7101d79153'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:32.847123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2c94ce39-2edc-4d69-a309-8e7101d79153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '796c5b646abb499d84b6fd9f7db1704f', 'request_id': 'req-2c94ce39-2edc-4d69-a309-8e7101d79153'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:32.953121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2c94ce39-2edc-4d69-a309-8e7101d79153 admin admin] The token cache is being invalidate because project 796c5b646abb499d84b6fd9f7db1704f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:00:32.954350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2c94ce39-2edc-4d69-a309-8e7101d79153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:32.954881 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2c94ce39-2edc-4d69-a309-8e7101d79153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:00:32.956675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 748/1510] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:00:32 2022] DELETE /identity/v3/projects/796c5b646abb499d84b6fd9f7db1704f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:00:42.087377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:00:42.948826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:34.419358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44c27e-605d-4237-bed6-ed97cf90577f None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:34.420320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44c27e-605d-4237-bed6-ed97cf90577f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:34.420320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44c27e-605d-4237-bed6-ed97cf90577f None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:34.421316 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 15:02:34.427260 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa8513-3011-4ccd-9a2d-90d316ba2bef None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:34.427917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa8513-3011-4ccd-9a2d-90d316ba2bef None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:34.427917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa8513-3011-4ccd-9a2d-90d316ba2bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:34.450626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-01fa8513-3011-4ccd-9a2d-90d316ba2bef None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:34.451934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-01fa8513-3011-4ccd-9a2d-90d316ba2bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:34.538161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:02:34.712581 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44c27e-605d-4237-bed6-ed97cf90577f None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:34.714779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-bc44c27e-605d-4237-bed6-ed97cf90577f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:34.840992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44c27e-605d-4237-bed6-ed97cf90577f None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:34.847502 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac859e78-cb6b-4966-871b-679a5ffbf1ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:34.848187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859e78-cb6b-4966-871b-679a5ffbf1ec service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:34.848469 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859e78-cb6b-4966-871b-679a5ffbf1ec service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:34.848744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859e78-cb6b-4966-871b-679a5ffbf1ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:34.858662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac859e78-cb6b-4966-871b-679a5ffbf1ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:34.860907 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac859e78-cb6b-4966-871b-679a5ffbf1ec service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:34.871638 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 764/1513] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:35.763856 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb1be0-f187-4557-9216-c0383c250696 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:35.764437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb1be0-f187-4557-9216-c0383c250696 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:35.764437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb1be0-f187-4557-9216-c0383c250696 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:35.766816 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:02:35.773142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b8e3-8b88-4a9f-a40a-fd08fd28bfef None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:35.773908 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b8e3-8b88-4a9f-a40a-fd08fd28bfef None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:35.774054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b8e3-8b88-4a9f-a40a-fd08fd28bfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:35.799081 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-ff38b8e3-8b88-4a9f-a40a-fd08fd28bfef None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:35.800387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ff38b8e3-8b88-4a9f-a40a-fd08fd28bfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:35.809696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:35.973091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb1be0-f187-4557-9216-c0383c250696 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:35.993166 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb1be0-f187-4557-9216-c0383c250696 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:36.004899 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb34cde-a478-4957-b8b4-aad1402d2405 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:36.004899 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb34cde-a478-4957-b8b4-aad1402d2405 service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:36.004899 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb34cde-a478-4957-b8b4-aad1402d2405 service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:36.004899 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb34cde-a478-4957-b8b4-aad1402d2405 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:36.014240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb34cde-a478-4957-b8b4-aad1402d2405 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:36.017500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb34cde-a478-4957-b8b4-aad1402d2405 service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:36.033986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 751/1516] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:02:37.271596 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29730172-4ff6-4001-9acb-555b993f4923 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:37.272746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29730172-4ff6-4001-9acb-555b993f4923 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:37.272746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29730172-4ff6-4001-9acb-555b993f4923 None None] PATH_INFO: `/v3` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:37.274220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 766/1517] 10.222.0.102 () {54 vars in 926 bytes} [Tue Jul 26 15:02:37 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:02:37.281310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e6d67-132f-48ad-bb51-0c4921db1d1b None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:37.282169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e6d67-132f-48ad-bb51-0c4921db1d1b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:37.282169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e6d67-132f-48ad-bb51-0c4921db1d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:37.342185 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-494e6d67-132f-48ad-bb51-0c4921db1d1b None None] MFA Rules not processed for user `42a0728e2f6249bdb45d6f44fb15aa4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:37.343505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-494e6d67-132f-48ad-bb51-0c4921db1d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:37.429644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 752/1518] 10.222.0.102 () {58 vars in 1016 bytes} [Tue Jul 26 15:02:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:41.970167 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662b445-dd88-45c6-9e1a-67ec9d492382 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:41.971040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662b445-dd88-45c6-9e1a-67ec9d492382 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:41.971040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662b445-dd88-45c6-9e1a-67ec9d492382 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:41.972073 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 767/1519] 10.222.0.94 () {54 vars in 961 bytes} [Tue Jul 26 15:02:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:02:41.978543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed915b-983e-4446-9da4-088e6bf50de5 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:41.979180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed915b-983e-4446-9da4-088e6bf50de5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:41.979288 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed915b-983e-4446-9da4-088e6bf50de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:42.001130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-59ed915b-983e-4446-9da4-088e6bf50de5 None None] MFA Rules not processed for user `42a0728e2f6249bdb45d6f44fb15aa4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:42.002647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-59ed915b-983e-4446-9da4-088e6bf50de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:42.015012 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 753/1520] 10.222.0.94 () {58 vars in 1058 bytes} [Tue Jul 26 15:02:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:42.025616 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662b445-dd88-45c6-9e1a-67ec9d492382 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:42.050011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662b445-dd88-45c6-9e1a-67ec9d492382 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:42.057638 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09754b-f375-4dd2-af4c-afc511789fcc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a0728e2f6249bdb45d6f44fb15aa4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:42.058650 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09754b-f375-4dd2-af4c-afc511789fcc service placement] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:42.059127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09754b-f375-4dd2-af4c-afc511789fcc service placement] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:42.059642 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09754b-f375-4dd2-af4c-afc511789fcc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:42.070725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09754b-f375-4dd2-af4c-afc511789fcc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:42.072217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09754b-f375-4dd2-af4c-afc511789fcc service placement] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:42.084434 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 768/1521] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:47.370184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03b047-69c3-4faf-82b3-34dffc14dc81 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:47.371052 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03b047-69c3-4faf-82b3-34dffc14dc81 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:47.371052 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03b047-69c3-4faf-82b3-34dffc14dc81 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:47.372601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:02:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 15:02:47.378578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2da63e-34ce-471a-8a49-1259c811aad9 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:47.379203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2da63e-34ce-471a-8a49-1259c811aad9 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:47.379203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2da63e-34ce-471a-8a49-1259c811aad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:47.528563 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-3c2da63e-34ce-471a-8a49-1259c811aad9 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:47.529936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3c2da63e-34ce-471a-8a49-1259c811aad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:47.539099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:47.702967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03b047-69c3-4faf-82b3-34dffc14dc81 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:47.720808 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03b047-69c3-4faf-82b3-34dffc14dc81 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:47.727104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8db9bc-f849-4ff2-b32d-d6e2d3f0ee42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:47.728171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8db9bc-f849-4ff2-b32d-d6e2d3f0ee42 service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:47.728595 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8db9bc-f849-4ff2-b32d-d6e2d3f0ee42 service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:47.728906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8db9bc-f849-4ff2-b32d-d6e2d3f0ee42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:47.738453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8db9bc-f849-4ff2-b32d-d6e2d3f0ee42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:47.740150 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8db9bc-f849-4ff2-b32d-d6e2d3f0ee42 service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:47.752599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 755/1524] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:48.701374 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c408eb-e5a7-4951-8ed4-af9726950090 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:48.702112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c408eb-e5a7-4951-8ed4-af9726950090 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:48.702112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c408eb-e5a7-4951-8ed4-af9726950090 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:48.703491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 15:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:02:48.709715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b691b-a725-4eff-9737-470878bc34b5 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:48.710242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b691b-a725-4eff-9737-470878bc34b5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:48.710406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b691b-a725-4eff-9737-470878bc34b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:48.732703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-d22b691b-a725-4eff-9737-470878bc34b5 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:48.733994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d22b691b-a725-4eff-9737-470878bc34b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:48.744786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 15:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:02:49.030618 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c408eb-e5a7-4951-8ed4-af9726950090 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:49.047966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c408eb-e5a7-4951-8ed4-af9726950090 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:49.053938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d0d72-a6dc-40a1-abe6-c38e80edbbc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:49.054571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d0d72-a6dc-40a1-abe6-c38e80edbbc2 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:49.054860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d0d72-a6dc-40a1-abe6-c38e80edbbc2 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:49.055127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d0d72-a6dc-40a1-abe6-c38e80edbbc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:49.065929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d0d72-a6dc-40a1-abe6-c38e80edbbc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:49.067641 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d0d72-a6dc-40a1-abe6-c38e80edbbc2 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:49.078938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 771/1527] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:02:53.487213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3e854-2ebd-4273-817c-75c805fcfff1 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:53.488128 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3e854-2ebd-4273-817c-75c805fcfff1 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:53.488128 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3e854-2ebd-4273-817c-75c805fcfff1 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:53.489658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 15:02:53.497104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f1313-16ac-4539-b162-f811eb02aa01 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:53.497979 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f1313-16ac-4539-b162-f811eb02aa01 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:53.497979 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f1313-16ac-4539-b162-f811eb02aa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:53.523154 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-2f9f1313-16ac-4539-b162-f811eb02aa01 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:53.524591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2f9f1313-16ac-4539-b162-f811eb02aa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:53.536469 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:02:53.629239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3e854-2ebd-4273-817c-75c805fcfff1 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:53.631398 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-70a3e854-2ebd-4273-817c-75c805fcfff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:53.681975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:02:53.741363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3e854-2ebd-4273-817c-75c805fcfff1 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:53.748116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd477b-0934-4375-94de-2d941f510d38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:53.748859 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd477b-0934-4375-94de-2d941f510d38 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:53.749190 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd477b-0934-4375-94de-2d941f510d38 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:53.749663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd477b-0934-4375-94de-2d941f510d38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:53.763117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd477b-0934-4375-94de-2d941f510d38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:53.774620 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd477b-0934-4375-94de-2d941f510d38 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:53.776994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 758/1530] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:02:55.954455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c39896-0767-4483-b0bc-0f2816418067 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:55.954987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c39896-0767-4483-b0bc-0f2816418067 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:55.955078 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c39896-0767-4483-b0bc-0f2816418067 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:55.956492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:02:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:02:55.964993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d637a-d8f7-4814-8939-b427f8ec127a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:55.965823 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d637a-d8f7-4814-8939-b427f8ec127a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:55.966025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d637a-d8f7-4814-8939-b427f8ec127a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:55.989896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-b80d637a-d8f7-4814-8939-b427f8ec127a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:55.991137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b80d637a-d8f7-4814-8939-b427f8ec127a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:55.999536 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:02:56.040342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c39896-0767-4483-b0bc-0f2816418067 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:02:56.052757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c39896-0767-4483-b0bc-0f2816418067 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:02:56.056513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b146a0-4ff0-4a62-b44a-8fdf6a78a588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:02:56.057188 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b146a0-4ff0-4a62-b44a-8fdf6a78a588 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:56.057500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b146a0-4ff0-4a62-b44a-8fdf6a78a588 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:56.057804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b146a0-4ff0-4a62-b44a-8fdf6a78a588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:56.069800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b146a0-4ff0-4a62-b44a-8fdf6a78a588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:02:56.071245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b146a0-4ff0-4a62-b44a-8fdf6a78a588 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:02:56.082897 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 774/1533] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:02:59.853449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393d86a-1577-4dd3-94e4-f0a5e2c58e2c None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:59.854070 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393d86a-1577-4dd3-94e4-f0a5e2c58e2c None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:59.854070 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393d86a-1577-4dd3-94e4-f0a5e2c58e2c None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:59.855350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 15:02:59.863042 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d596a1-968e-48f4-a1f4-b1919e1e134a None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:02:59.863714 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d596a1-968e-48f4-a1f4-b1919e1e134a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:02:59.864184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d596a1-968e-48f4-a1f4-b1919e1e134a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:02:59.898600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-99d596a1-968e-48f4-a1f4-b1919e1e134a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:02:59.900773 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-99d596a1-968e-48f4-a1f4-b1919e1e134a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:02:59.915292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:00.254622 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393d86a-1577-4dd3-94e4-f0a5e2c58e2c None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:00.256469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a393d86a-1577-4dd3-94e4-f0a5e2c58e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:00.322717 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393d86a-1577-4dd3-94e4-f0a5e2c58e2c None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:00.326409 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f95d9-d3a6-4406-907d-09c2eb5e89f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:00.327049 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f95d9-d3a6-4406-907d-09c2eb5e89f0 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:00.327327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f95d9-d3a6-4406-907d-09c2eb5e89f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:00.327643 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f95d9-d3a6-4406-907d-09c2eb5e89f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:00.338321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f95d9-d3a6-4406-907d-09c2eb5e89f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:00.339822 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f95d9-d3a6-4406-907d-09c2eb5e89f0 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:00.351978 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 761/1536] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:03:02.569236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc366972-289a-44d9-8772-5edc18bc871f None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:02.569835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc366972-289a-44d9-8772-5edc18bc871f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:02.569835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc366972-289a-44d9-8772-5edc18bc871f None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:02.571165 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 15:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:03:02.577351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76554b09-1726-42dd-8032-c72a465029be None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:02.578437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76554b09-1726-42dd-8032-c72a465029be None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:02.578647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76554b09-1726-42dd-8032-c72a465029be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:02.601132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-76554b09-1726-42dd-8032-c72a465029be None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:03:02.602381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-76554b09-1726-42dd-8032-c72a465029be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:02.611330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 15:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:03:02.945825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc366972-289a-44d9-8772-5edc18bc871f None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:03:02.947936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-cc366972-289a-44d9-8772-5edc18bc871f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:03:02.957428 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=77 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:02.965314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc366972-289a-44d9-8772-5edc18bc871f None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:03:02.969690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390ad5b-e2d4-4790-8952-823b50f754f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:03:02.975401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390ad5b-e2d4-4790-8952-823b50f754f9 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:03:02.975401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390ad5b-e2d4-4790-8952-823b50f754f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:03:02.975401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390ad5b-e2d4-4790-8952-823b50f754f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:03:02.981495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390ad5b-e2d4-4790-8952-823b50f754f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:03:02.982968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390ad5b-e2d4-4790-8952-823b50f754f9 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:03:02.994400 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 777/1539] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:03:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:03:12.594849 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=47 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:03:13.011847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=5 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:38.685360 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fefcb3-b7ad-4304-90d0-c7b119bb35d0 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:38.688035 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fefcb3-b7ad-4304-90d0-c7b119bb35d0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:38.688035 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fefcb3-b7ad-4304-90d0-c7b119bb35d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:38.716737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-62fefcb3-b7ad-4304-90d0-c7b119bb35d0 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:38.717993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-62fefcb3-b7ad-4304-90d0-c7b119bb35d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:38.727119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 763/1540] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:04:38.737745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167ec8f-3e64-4468-972d-17c799dbf55e None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:38.738930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167ec8f-3e64-4468-972d-17c799dbf55e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:38.739447 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167ec8f-3e64-4468-972d-17c799dbf55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:38.789803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-2167ec8f-3e64-4468-972d-17c799dbf55e None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:38.792165 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2167ec8f-3e64-4468-972d-17c799dbf55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:38.811859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 778/1541] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:04:38.817392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fefcb3-b7ad-4304-90d0-c7b119bb35d0 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:38.832945 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fefcb3-b7ad-4304-90d0-c7b119bb35d0 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:38.842098 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb2856-7d25-4cbc-b409-836124976ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:38.843142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb2856-7d25-4cbc-b409-836124976ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:38.843846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb2856-7d25-4cbc-b409-836124976ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:38.844678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb2856-7d25-4cbc-b409-836124976ea4 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:38.845792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb2856-7d25-4cbc-b409-836124976ea4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:04:38.846331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb2856-7d25-4cbc-b409-836124976ea4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:38.848739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb2856-7d25-4cbc-b409-836124976ea4 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:38.849711 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03007f9-4003-4432-9f16-9e7b7855397b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:38.850629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03007f9-4003-4432-9f16-9e7b7855397b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:38.851108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03007f9-4003-4432-9f16-9e7b7855397b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:38.872974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 764/1542] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:04:38.891255 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb2856-7d25-4cbc-b409-836124976ea4 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:38.897122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b03007f9-4003-4432-9f16-9e7b7855397b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:38.898568 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b03007f9-4003-4432-9f16-9e7b7855397b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:38.918685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb2856-7d25-4cbc-b409-836124976ea4 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:38.920395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 779/1543] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:38.921834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d7af4-14b1-41c1-8ef2-e60d777b8d78 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:38.922181 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d7af4-14b1-41c1-8ef2-e60d777b8d78 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:38.922600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d7af4-14b1-41c1-8ef2-e60d777b8d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:38.931316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd5e46-2401-4d1d-8a96-e39357544257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:38.932423 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd5e46-2401-4d1d-8a96-e39357544257 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:38.933549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd5e46-2401-4d1d-8a96-e39357544257 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:38.934239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd5e46-2401-4d1d-8a96-e39357544257 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:38.935311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd5e46-2401-4d1d-8a96-e39357544257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:38.938791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: WARNING py.warnings [None req-36fd5e46-2401-4d1d-8a96-e39357544257 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:04:38.938791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: warnings.warn(msg) Jul 26 15:04:38.938791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: Jul 26 15:04:38.940582 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd5e46-2401-4d1d-8a96-e39357544257 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:38.968374 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-139d7af4-14b1-41c1-8ef2-e60d777b8d78 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:38.970251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-139d7af4-14b1-41c1-8ef2-e60d777b8d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:38.996329 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 780/1544] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:38.998795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f94247-8669-4f5b-8910-fc7e4160a97b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:38.999085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f94247-8669-4f5b-8910-fc7e4160a97b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:38.999846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f94247-8669-4f5b-8910-fc7e4160a97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.046393 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-55f94247-8669-4f5b-8910-fc7e4160a97b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:39.048615 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-55f94247-8669-4f5b-8910-fc7e4160a97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:39.075097 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 781/1545] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:39.076485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced69572-eb0a-4ca7-9b66-d29cd4404691 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.076766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced69572-eb0a-4ca7-9b66-d29cd4404691 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.077035 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced69572-eb0a-4ca7-9b66-d29cd4404691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.099622 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-ced69572-eb0a-4ca7-9b66-d29cd4404691 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:39.100873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ced69572-eb0a-4ca7-9b66-d29cd4404691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:39.110445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 782/1546] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:39.111859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c238b-809e-4402-a3a0-ef6fa49f4ead None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.112295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c238b-809e-4402-a3a0-ef6fa49f4ead None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.112595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c238b-809e-4402-a3a0-ef6fa49f4ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.133843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-256c238b-809e-4402-a3a0-ef6fa49f4ead None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:39.135037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-256c238b-809e-4402-a3a0-ef6fa49f4ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:39.139514 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 765/1547] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:38 2022] POST /identity/v3/projects => generated 369 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:39.141415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513d320-3188-4c3c-8413-6ce9b4bea3e6 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.141705 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513d320-3188-4c3c-8413-6ce9b4bea3e6 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.141977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513d320-3188-4c3c-8413-6ce9b4bea3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.143678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 783/1548] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:39.150380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c238b-809e-4402-a3a0-ef6fa49f4ead None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:39.161143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c238b-809e-4402-a3a0-ef6fa49f4ead None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:39.181484 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-e513d320-3188-4c3c-8413-6ce9b4bea3e6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:39.184609 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e513d320-3188-4c3c-8413-6ce9b4bea3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:39.232187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e380458-4bd3-4672-b497-c2bf19bfeca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:39.232742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e380458-4bd3-4672-b497-c2bf19bfeca3 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.233015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e380458-4bd3-4672-b497-c2bf19bfeca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.233306 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e380458-4bd3-4672-b497-c2bf19bfeca3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.233997 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e380458-4bd3-4672-b497-c2bf19bfeca3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:04:39.234304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e380458-4bd3-4672-b497-c2bf19bfeca3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:39.235386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: WARNING py.warnings [None req-3e380458-4bd3-4672-b497-c2bf19bfeca3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:04:39.235386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: warnings.warn(msg) Jul 26 15:04:39.235386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: Jul 26 15:04:39.236305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e380458-4bd3-4672-b497-c2bf19bfeca3 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:39.252315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 784/1549] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:04:39.253589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850fb3a-a25a-425d-bf42-b07d9aa47efc None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.253928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850fb3a-a25a-425d-bf42-b07d9aa47efc None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.254182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850fb3a-a25a-425d-bf42-b07d9aa47efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.256218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 766/1550] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:39.257782 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663f1a5-1cf9-46df-8b84-38dc654e144c None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.258078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663f1a5-1cf9-46df-8b84-38dc654e144c None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.258359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663f1a5-1cf9-46df-8b84-38dc654e144c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.280119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-e663f1a5-1cf9-46df-8b84-38dc654e144c None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:39.281351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e663f1a5-1cf9-46df-8b84-38dc654e144c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:39.288458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-d850fb3a-a25a-425d-bf42-b07d9aa47efc None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:39.289730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 767/1551] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:39.290083 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d850fb3a-a25a-425d-bf42-b07d9aa47efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:39.290565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663f1a5-1cf9-46df-8b84-38dc654e144c None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:39.304053 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663f1a5-1cf9-46df-8b84-38dc654e144c None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:39.304328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 785/1552] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:39.305377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850fb3a-a25a-425d-bf42-b07d9aa47efc None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:39.310796 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae6ca6-62ff-461a-ae7e-91b5c5139ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:39.311429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6ca6-62ff-461a-ae7e-91b5c5139ede admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.311740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6ca6-62ff-461a-ae7e-91b5c5139ede admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.312046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6ca6-62ff-461a-ae7e-91b5c5139ede admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.313070 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae6ca6-62ff-461a-ae7e-91b5c5139ede admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:04:39.313380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae6ca6-62ff-461a-ae7e-91b5c5139ede admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:39.314985 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae6ca6-62ff-461a-ae7e-91b5c5139ede admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:39.316326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850fb3a-a25a-425d-bf42-b07d9aa47efc None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:39.321075 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c652d-0048-4a38-9398-43a710297a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:39.321603 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c652d-0048-4a38-9398-43a710297a90 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.321888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c652d-0048-4a38-9398-43a710297a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.322162 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c652d-0048-4a38-9398-43a710297a90 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.322949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c652d-0048-4a38-9398-43a710297a90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:39.324281 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: WARNING py.warnings [None req-118c652d-0048-4a38-9398-43a710297a90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:04:39.324281 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: warnings.warn(msg) Jul 26 15:04:39.324281 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: Jul 26 15:04:39.326173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c652d-0048-4a38-9398-43a710297a90 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:39.328510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 768/1553] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:04:39.329662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd50370-6d78-4059-a7fd-ba9705b2e35a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.329946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd50370-6d78-4059-a7fd-ba9705b2e35a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.330224 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd50370-6d78-4059-a7fd-ba9705b2e35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.353521 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-3bd50370-6d78-4059-a7fd-ba9705b2e35a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:39.356569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3bd50370-6d78-4059-a7fd-ba9705b2e35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:39.364635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 769/1554] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:39.366047 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1add9c0-d295-4779-897b-e4d8b6167c9d None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.366275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1add9c0-d295-4779-897b-e4d8b6167c9d None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.366551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1add9c0-d295-4779-897b-e4d8b6167c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.388200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-e1add9c0-d295-4779-897b-e4d8b6167c9d None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:39.389535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e1add9c0-d295-4779-897b-e4d8b6167c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:39.397838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 770/1555] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:39.398677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1add9c0-d295-4779-897b-e4d8b6167c9d None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:39.409388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1add9c0-d295-4779-897b-e4d8b6167c9d None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:39.414427 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d030e-4523-4ff3-877f-61ecf16d42f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:39.415070 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d030e-4523-4ff3-877f-61ecf16d42f3 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.415357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d030e-4523-4ff3-877f-61ecf16d42f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.415664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d030e-4523-4ff3-877f-61ecf16d42f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.416352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d030e-4523-4ff3-877f-61ecf16d42f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:39.419186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d030e-4523-4ff3-877f-61ecf16d42f3 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:39.500325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 786/1556] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/users => generated 285 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:39.500325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4f7d1-48a1-4dde-b9d7-e035a14a7268 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.500325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4f7d1-48a1-4dde-b9d7-e035a14a7268 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.500325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4f7d1-48a1-4dde-b9d7-e035a14a7268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.523919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-f2d4f7d1-48a1-4dde-b9d7-e035a14a7268 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:39.525124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f2d4f7d1-48a1-4dde-b9d7-e035a14a7268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:39.534236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 787/1557] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:39.535321 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f580f70-a8e2-495c-b08c-6b2e352e78f9 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.535659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f580f70-a8e2-495c-b08c-6b2e352e78f9 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.535987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f580f70-a8e2-495c-b08c-6b2e352e78f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.555939 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-9f580f70-a8e2-495c-b08c-6b2e352e78f9 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:39.557203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9f580f70-a8e2-495c-b08c-6b2e352e78f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:39.565841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 788/1558] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:39.567102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8dc00-503e-4934-9a2d-d4db95888691 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.567382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8dc00-503e-4934-9a2d-d4db95888691 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.567750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8dc00-503e-4934-9a2d-d4db95888691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.589713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-3ab8dc00-503e-4934-9a2d-d4db95888691 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:39.592401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 771/1559] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/projects => generated 357 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:39.598879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3ab8dc00-503e-4934-9a2d-d4db95888691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:39.599027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d030e-4523-4ff3-877f-61ecf16d42f3 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:39.659616 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 789/1560] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:39.660478 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8dc00-503e-4934-9a2d-d4db95888691 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:39.668216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d030e-4523-4ff3-877f-61ecf16d42f3 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:39.671031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8dc00-503e-4934-9a2d-d4db95888691 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:39.677264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5955a-314f-4637-8c13-dc79211a4a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:39.677264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5955a-314f-4637-8c13-dc79211a4a10 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.677264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5955a-314f-4637-8c13-dc79211a4a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.677264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5955a-314f-4637-8c13-dc79211a4a10 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.677652 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5955a-314f-4637-8c13-dc79211a4a10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:39.677754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb435e5-7aae-435c-ae9b-e078eb7833cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:39.677754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb435e5-7aae-435c-ae9b-e078eb7833cf admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.677754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb435e5-7aae-435c-ae9b-e078eb7833cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.677754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb435e5-7aae-435c-ae9b-e078eb7833cf admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.678135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5955a-314f-4637-8c13-dc79211a4a10 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:39.678239 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb435e5-7aae-435c-ae9b-e078eb7833cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:04:39.678239 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb435e5-7aae-435c-ae9b-e078eb7833cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:39.679210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb435e5-7aae-435c-ae9b-e078eb7833cf admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:39.687922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:39.691944 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 790/1561] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:04:39.700044 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c4448-7581-4370-a3a3-ad31cffb93f9 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.700044 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c4448-7581-4370-a3a3-ad31cffb93f9 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.700044 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c4448-7581-4370-a3a3-ad31cffb93f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.703339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:39.713865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7c9c4448-7581-4370-a3a3-ad31cffb93f9 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:39.715099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7c9c4448-7581-4370-a3a3-ad31cffb93f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:39.722818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 791/1562] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:39.723768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c4448-7581-4370-a3a3-ad31cffb93f9 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:39.732922 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c4448-7581-4370-a3a3-ad31cffb93f9 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:39.737684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b0866-a9dd-4bbd-bdc7-04770f744b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:39.738215 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b0866-a9dd-4bbd-bdc7-04770f744b89 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.738786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b0866-a9dd-4bbd-bdc7-04770f744b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.738786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b0866-a9dd-4bbd-bdc7-04770f744b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.739404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b0866-a9dd-4bbd-bdc7-04770f744b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:39.740995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b0866-a9dd-4bbd-bdc7-04770f744b89 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:39.752202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 792/1563] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:39.753061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b0866-a9dd-4bbd-bdc7-04770f744b89 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:39.763380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b0866-a9dd-4bbd-bdc7-04770f744b89 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:39.776440 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aefbb4-6735-45bb-a970-8c7d3923fb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:39.776440 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aefbb4-6735-45bb-a970-8c7d3923fb12 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.776440 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aefbb4-6735-45bb-a970-8c7d3923fb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.776440 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aefbb4-6735-45bb-a970-8c7d3923fb12 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.777005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aefbb4-6735-45bb-a970-8c7d3923fb12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:04:39.777005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aefbb4-6735-45bb-a970-8c7d3923fb12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:39.777005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aefbb4-6735-45bb-a970-8c7d3923fb12 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:39.784501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 793/1564] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:04:39.785294 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aefbb4-6735-45bb-a970-8c7d3923fb12 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:39.794432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aefbb4-6735-45bb-a970-8c7d3923fb12 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:39.799259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0f020-6f8f-4ac5-a135-c876107eef2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:39.799983 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0f020-6f8f-4ac5-a135-c876107eef2e admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.800268 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0f020-6f8f-4ac5-a135-c876107eef2e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.800542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0f020-6f8f-4ac5-a135-c876107eef2e admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.801238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0f020-6f8f-4ac5-a135-c876107eef2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:04:39.801540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0f020-6f8f-4ac5-a135-c876107eef2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:39.803130 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0f020-6f8f-4ac5-a135-c876107eef2e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:39.809454 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 772/1565] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/projects => generated 377 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:39.809454 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5955a-314f-4637-8c13-dc79211a4a10 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:39.817219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 794/1566] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:04:39.820402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0f020-6f8f-4ac5-a135-c876107eef2e admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:39.831535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0f020-6f8f-4ac5-a135-c876107eef2e admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:39.873020 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5955a-314f-4637-8c13-dc79211a4a10 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:39.878018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409968c-9f2c-462d-bba4-e291c7fbcff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:39.878641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409968c-9f2c-462d-bba4-e291c7fbcff0 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.878921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409968c-9f2c-462d-bba4-e291c7fbcff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.879191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409968c-9f2c-462d-bba4-e291c7fbcff0 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.879922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409968c-9f2c-462d-bba4-e291c7fbcff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:39.881891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409968c-9f2c-462d-bba4-e291c7fbcff0 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:39.894597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc492bce-40d8-4298-887c-6f6cf7b64875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:39.895446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc492bce-40d8-4298-887c-6f6cf7b64875 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.895778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc492bce-40d8-4298-887c-6f6cf7b64875 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.896098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc492bce-40d8-4298-887c-6f6cf7b64875 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.896802 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc492bce-40d8-4298-887c-6f6cf7b64875 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:04:39.897108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc492bce-40d8-4298-887c-6f6cf7b64875 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:39.898847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc492bce-40d8-4298-887c-6f6cf7b64875 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:39.911604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 795/1567] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:04:39.912396 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc492bce-40d8-4298-887c-6f6cf7b64875 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:39.922266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc492bce-40d8-4298-887c-6f6cf7b64875 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:39.927066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb9c02-a7d8-444d-9ce0-5503c4dc5993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:39.927782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb9c02-a7d8-444d-9ce0-5503c4dc5993 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:39.928066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb9c02-a7d8-444d-9ce0-5503c4dc5993 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:39.928335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb9c02-a7d8-444d-9ce0-5503c4dc5993 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:39.930343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb9c02-a7d8-444d-9ce0-5503c4dc5993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:39.930979 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb9c02-a7d8-444d-9ce0-5503c4dc5993 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:40.021781 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 773/1568] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/users => generated 279 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:40.024134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409968c-9f2c-462d-bba4-e291c7fbcff0 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:40.034465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409968c-9f2c-462d-bba4-e291c7fbcff0 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:40.039551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7be0bd-86ea-48f5-bdcf-eac795e0005b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:40.040163 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7be0bd-86ea-48f5-bdcf-eac795e0005b admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:40.040444 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7be0bd-86ea-48f5-bdcf-eac795e0005b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:40.040724 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7be0bd-86ea-48f5-bdcf-eac795e0005b admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:40.041419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7be0bd-86ea-48f5-bdcf-eac795e0005b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:04:40.041720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7be0bd-86ea-48f5-bdcf-eac795e0005b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:40.044163 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7be0bd-86ea-48f5-bdcf-eac795e0005b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:40.060058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 774/1569] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:04:40.060944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7be0bd-86ea-48f5-bdcf-eac795e0005b admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:40.071285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7be0bd-86ea-48f5-bdcf-eac795e0005b admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:40.076865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcf818-4525-4747-884f-bb4a764d1522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:40.077549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcf818-4525-4747-884f-bb4a764d1522 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:40.077997 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcf818-4525-4747-884f-bb4a764d1522 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:40.078280 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcf818-4525-4747-884f-bb4a764d1522 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:40.078989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbcf818-4525-4747-884f-bb4a764d1522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:40.080690 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbcf818-4525-4747-884f-bb4a764d1522 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:40.093022 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 775/1570] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:40.093513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcf818-4525-4747-884f-bb4a764d1522 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:40.102198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 796/1571] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:39 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:40.103223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb9c02-a7d8-444d-9ce0-5503c4dc5993 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:40.108302 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcf818-4525-4747-884f-bb4a764d1522 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:40.171920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f08ec-ef95-4a16-ba7d-7bb402a43af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:40.172599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f08ec-ef95-4a16-ba7d-7bb402a43af0 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:40.172894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f08ec-ef95-4a16-ba7d-7bb402a43af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:40.173169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f08ec-ef95-4a16-ba7d-7bb402a43af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:40.173849 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f08ec-ef95-4a16-ba7d-7bb402a43af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:40.173952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb9c02-a7d8-444d-9ce0-5503c4dc5993 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:40.175891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f08ec-ef95-4a16-ba7d-7bb402a43af0 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:40.178871 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a2147-ba87-4d9c-81d7-f529097635e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:40.179542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a2147-ba87-4d9c-81d7-f529097635e3 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:40.179828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a2147-ba87-4d9c-81d7-f529097635e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:40.180128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a2147-ba87-4d9c-81d7-f529097635e3 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:40.180790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a2147-ba87-4d9c-81d7-f529097635e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:40.182843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a2147-ba87-4d9c-81d7-f529097635e3 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:40.351376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 776/1572] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:40 2022] POST /identity/v3/projects => generated 383 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:40.353174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f08ec-ef95-4a16-ba7d-7bb402a43af0 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:40.391934 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 797/1573] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:40 2022] POST /identity/v3/users => generated 289 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:40.391934 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a2147-ba87-4d9c-81d7-f529097635e3 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:40.402928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a2147-ba87-4d9c-81d7-f529097635e3 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:40.404435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:40.424920 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:40.431060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f08ec-ef95-4a16-ba7d-7bb402a43af0 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:40.438463 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ad5b0-6478-4fa9-a748-54d139b10a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:40.439121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ad5b0-6478-4fa9-a748-54d139b10a73 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:40.439400 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ad5b0-6478-4fa9-a748-54d139b10a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:40.439698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ad5b0-6478-4fa9-a748-54d139b10a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:40.440371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ad5b0-6478-4fa9-a748-54d139b10a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:40.442280 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ad5b0-6478-4fa9-a748-54d139b10a73 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:40.470973 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eabf83-2e1b-4ec9-ba3d-50bfce6fac3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:40.471702 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eabf83-2e1b-4ec9-ba3d-50bfce6fac3f admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:40.471982 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eabf83-2e1b-4ec9-ba3d-50bfce6fac3f admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:40.472250 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eabf83-2e1b-4ec9-ba3d-50bfce6fac3f admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:40.472911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eabf83-2e1b-4ec9-ba3d-50bfce6fac3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:40.475002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eabf83-2e1b-4ec9-ba3d-50bfce6fac3f admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:40.563057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 777/1574] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:40 2022] POST /identity/v3/projects => generated 377 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:40.564058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ad5b0-6478-4fa9-a748-54d139b10a73 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:40.598365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 798/1575] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:40 2022] POST /identity/v3/projects => generated 391 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:40.599257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eabf83-2e1b-4ec9-ba3d-50bfce6fac3f admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:40.645552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ad5b0-6478-4fa9-a748-54d139b10a73 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:40.679062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eabf83-2e1b-4ec9-ba3d-50bfce6fac3f admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:40.685554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9a542-bcfd-45ae-b68c-023044c3a22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:40.686231 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9a542-bcfd-45ae-b68c-023044c3a22c admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:40.686540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9a542-bcfd-45ae-b68c-023044c3a22c admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:40.686778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9a542-bcfd-45ae-b68c-023044c3a22c admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:40.687490 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9a542-bcfd-45ae-b68c-023044c3a22c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:40.689494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9a542-bcfd-45ae-b68c-023044c3a22c admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:40.720390 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37477b2e-25ab-4bde-a81e-6ad6fdb5b28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:40.721062 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37477b2e-25ab-4bde-a81e-6ad6fdb5b28f admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:40.721272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37477b2e-25ab-4bde-a81e-6ad6fdb5b28f admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:40.721544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37477b2e-25ab-4bde-a81e-6ad6fdb5b28f admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:40.722231 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37477b2e-25ab-4bde-a81e-6ad6fdb5b28f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:40.723936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37477b2e-25ab-4bde-a81e-6ad6fdb5b28f admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:40.735074 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 778/1576] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:40.739408 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37477b2e-25ab-4bde-a81e-6ad6fdb5b28f admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:40.746789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37477b2e-25ab-4bde-a81e-6ad6fdb5b28f admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:40.752346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5c5ab-ed97-476a-b96b-77263103f6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:40.753032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c5ab-ed97-476a-b96b-77263103f6db admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:40.753309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c5ab-ed97-476a-b96b-77263103f6db admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:40.753583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5c5ab-ed97-476a-b96b-77263103f6db admin admin] PATH_INFO: `/v3/projects/d1d1829007bb46e380944cf9e32fef49/users/8ce0f05db1ba478caa48f08661bc34a3/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:40.770023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5c5ab-ed97-476a-b96b-77263103f6db admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d1829007bb46e380944cf9e32fef49, user_id=8ce0f05db1ba478caa48f08661bc34a3, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:40.771023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: WARNING py.warnings [None req-bec5c5ab-ed97-476a-b96b-77263103f6db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:04:40.771023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: warnings.warn(msg) Jul 26 15:04:40.771023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: Jul 26 15:04:40.773161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5c5ab-ed97-476a-b96b-77263103f6db admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:40.836669 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 799/1577] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:40 2022] POST /identity/v3/projects => generated 365 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:40.837557 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9a542-bcfd-45ae-b68c-023044c3a22c admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:40.914399 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9a542-bcfd-45ae-b68c-023044c3a22c admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:40.916299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 779/1578] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:40 2022] PUT /identity/v3/projects/d1d1829007bb46e380944cf9e32fef49/users/8ce0f05db1ba478caa48f08661bc34a3/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:40.918312 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5c5ab-ed97-476a-b96b-77263103f6db admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:40.963907 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:40.986183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:40.986509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7bbd43-db47-47f0-b69b-5f8606c5a810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:40.987145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bbd43-db47-47f0-b69b-5f8606c5a810 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:40.987426 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bbd43-db47-47f0-b69b-5f8606c5a810 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:40.987748 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bbd43-db47-47f0-b69b-5f8606c5a810 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:40.988452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7bbd43-db47-47f0-b69b-5f8606c5a810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:40.990476 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7bbd43-db47-47f0-b69b-5f8606c5a810 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.006256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5c5ab-ed97-476a-b96b-77263103f6db admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.013298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00421b-f554-4539-b5df-92107bf8c7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.013930 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00421b-f554-4539-b5df-92107bf8c7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.014216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00421b-f554-4539-b5df-92107bf8c7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.014491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00421b-f554-4539-b5df-92107bf8c7a9 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.015180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00421b-f554-4539-b5df-92107bf8c7a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.017302 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00421b-f554-4539-b5df-92107bf8c7a9 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.145222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 800/1579] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:40 2022] POST /identity/v3/users => generated 293 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:41.146223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7bbd43-db47-47f0-b69b-5f8606c5a810 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.157428 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7bbd43-db47-47f0-b69b-5f8606c5a810 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.162278 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f549a6-172c-4ad1-b8ea-d4d897aeb7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.162933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f549a6-172c-4ad1-b8ea-d4d897aeb7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.163258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f549a6-172c-4ad1-b8ea-d4d897aeb7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.163691 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f549a6-172c-4ad1-b8ea-d4d897aeb7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.164459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f549a6-172c-4ad1-b8ea-d4d897aeb7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.165996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f549a6-172c-4ad1-b8ea-d4d897aeb7b6 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.180667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 801/1580] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:41.181655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f549a6-172c-4ad1-b8ea-d4d897aeb7b6 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.186242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 780/1581] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:40 2022] POST /identity/v3/users => generated 292 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:41.186242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00421b-f554-4539-b5df-92107bf8c7a9 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.192804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f549a6-172c-4ad1-b8ea-d4d897aeb7b6 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.198122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7d847-688e-47ae-a592-9815a3c760af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.198424 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00421b-f554-4539-b5df-92107bf8c7a9 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.198830 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7d847-688e-47ae-a592-9815a3c760af admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.199313 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7d847-688e-47ae-a592-9815a3c760af admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.199821 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7d847-688e-47ae-a592-9815a3c760af admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.200506 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7d847-688e-47ae-a592-9815a3c760af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.202569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7d847-688e-47ae-a592-9815a3c760af admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.203639 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08decbc6-d80a-4eb3-9af5-af407f26311f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.204346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08decbc6-d80a-4eb3-9af5-af407f26311f admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.204675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08decbc6-d80a-4eb3-9af5-af407f26311f admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.204970 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08decbc6-d80a-4eb3-9af5-af407f26311f admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.205677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08decbc6-d80a-4eb3-9af5-af407f26311f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.207661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08decbc6-d80a-4eb3-9af5-af407f26311f admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.303436 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 802/1583] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:41 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:41.303838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 781/1583] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:41 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:41.303930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7d847-688e-47ae-a592-9815a3c760af admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.304778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08decbc6-d80a-4eb3-9af5-af407f26311f admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.314407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7d847-688e-47ae-a592-9815a3c760af admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.317359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08decbc6-d80a-4eb3-9af5-af407f26311f admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.319109 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc370354-1706-4629-b8ad-1d8c6095060a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.319761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc370354-1706-4629-b8ad-1d8c6095060a admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.320035 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc370354-1706-4629-b8ad-1d8c6095060a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.320297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc370354-1706-4629-b8ad-1d8c6095060a admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.320982 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc370354-1706-4629-b8ad-1d8c6095060a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.322566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc370354-1706-4629-b8ad-1d8c6095060a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.322675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346e661-8e8a-4a74-b33c-6a7a411716b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.323290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346e661-8e8a-4a74-b33c-6a7a411716b4 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.323707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346e661-8e8a-4a74-b33c-6a7a411716b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.324348 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346e661-8e8a-4a74-b33c-6a7a411716b4 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.325405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346e661-8e8a-4a74-b33c-6a7a411716b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.327792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346e661-8e8a-4a74-b33c-6a7a411716b4 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.334837 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 803/1584] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:41.336358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1d6ee-7339-4858-9c9e-370ab1bd7b72 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.336498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1d6ee-7339-4858-9c9e-370ab1bd7b72 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.336732 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1d6ee-7339-4858-9c9e-370ab1bd7b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.367853 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-a8f1d6ee-7339-4858-9c9e-370ab1bd7b72 None None] MFA Rules not processed for user `8ce0f05db1ba478caa48f08661bc34a3`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:41.369137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a8f1d6ee-7339-4858-9c9e-370ab1bd7b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:41.408355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 782/1585] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:41 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:41.412246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346e661-8e8a-4a74-b33c-6a7a411716b4 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.424464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346e661-8e8a-4a74-b33c-6a7a411716b4 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.429824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b5ae8-181c-4547-acbc-d09bf349ca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.430599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b5ae8-181c-4547-acbc-d09bf349ca1c admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.430982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b5ae8-181c-4547-acbc-d09bf349ca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.431348 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b5ae8-181c-4547-acbc-d09bf349ca1c admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.432243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b5ae8-181c-4547-acbc-d09bf349ca1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.434106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b5ae8-181c-4547-acbc-d09bf349ca1c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.444383 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 804/1586] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:41.445762 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 783/1587] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:41.446749 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b5ae8-181c-4547-acbc-d09bf349ca1c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.449511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1d6ee-7339-4858-9c9e-370ab1bd7b72 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.459796 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b5ae8-181c-4547-acbc-d09bf349ca1c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.460552 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1d6ee-7339-4858-9c9e-370ab1bd7b72 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.465809 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468737b-b7ad-4647-9fea-56ad450b0c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.466257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b13dc3-6c46-4d1c-bcd4-0cfc9154f4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.466529 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468737b-b7ad-4647-9fea-56ad450b0c85 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.466713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468737b-b7ad-4647-9fea-56ad450b0c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.466983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468737b-b7ad-4647-9fea-56ad450b0c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.467187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b13dc3-6c46-4d1c-bcd4-0cfc9154f4dd admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.467567 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b13dc3-6c46-4d1c-bcd4-0cfc9154f4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.467726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468737b-b7ad-4647-9fea-56ad450b0c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.467948 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b13dc3-6c46-4d1c-bcd4-0cfc9154f4dd admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.468988 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b13dc3-6c46-4d1c-bcd4-0cfc9154f4dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.469310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468737b-b7ad-4647-9fea-56ad450b0c85 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.470653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b13dc3-6c46-4d1c-bcd4-0cfc9154f4dd admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.481290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 784/1588] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:41.482239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468737b-b7ad-4647-9fea-56ad450b0c85 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.492577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 805/1589] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:41.492577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b13dc3-6c46-4d1c-bcd4-0cfc9154f4dd admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.493449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468737b-b7ad-4647-9fea-56ad450b0c85 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.498870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c071d-a15d-4740-9fab-448b4a1eb2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.499770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c071d-a15d-4740-9fab-448b4a1eb2cb admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.499897 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b13dc3-6c46-4d1c-bcd4-0cfc9154f4dd admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.500054 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c071d-a15d-4740-9fab-448b4a1eb2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.500325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c071d-a15d-4740-9fab-448b4a1eb2cb admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.501024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c071d-a15d-4740-9fab-448b4a1eb2cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.502624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c071d-a15d-4740-9fab-448b4a1eb2cb admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.504819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0518dd3-1b45-4730-978e-849ea04698fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.505443 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0518dd3-1b45-4730-978e-849ea04698fa admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.505726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0518dd3-1b45-4730-978e-849ea04698fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.506024 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0518dd3-1b45-4730-978e-849ea04698fa admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.506692 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0518dd3-1b45-4730-978e-849ea04698fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.508335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0518dd3-1b45-4730-978e-849ea04698fa admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.514947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 785/1590] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:41.516272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c071d-a15d-4740-9fab-448b4a1eb2cb admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.522672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 806/1591] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:41.522672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0518dd3-1b45-4730-978e-849ea04698fa admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.529645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c071d-a15d-4740-9fab-448b4a1eb2cb admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.536378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f444e-7f3e-4232-9ac9-cc218acdd3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.537068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f444e-7f3e-4232-9ac9-cc218acdd3a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.537352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f444e-7f3e-4232-9ac9-cc218acdd3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.537626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f444e-7f3e-4232-9ac9-cc218acdd3a5 admin admin] PATH_INFO: `/v3/projects/3d04940b01354bcabce0a75683732b0d/users/3c5d9ba77968401da3c87d08b6b6027b/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.538205 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0518dd3-1b45-4730-978e-849ea04698fa admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.543373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c20b6a-96e4-4bad-9742-a0157fccee74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.551738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20b6a-96e4-4bad-9742-a0157fccee74 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.551738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20b6a-96e4-4bad-9742-a0157fccee74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.551738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20b6a-96e4-4bad-9742-a0157fccee74 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.551738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c20b6a-96e4-4bad-9742-a0157fccee74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.551738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c20b6a-96e4-4bad-9742-a0157fccee74 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.554749 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f444e-7f3e-4232-9ac9-cc218acdd3a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d04940b01354bcabce0a75683732b0d, user_id=3c5d9ba77968401da3c87d08b6b6027b, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.557435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f444e-7f3e-4232-9ac9-cc218acdd3a5 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.559262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 807/1592] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:41.560017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c20b6a-96e4-4bad-9742-a0157fccee74 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.577795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c20b6a-96e4-4bad-9742-a0157fccee74 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.584202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee461ac-6432-4d5a-a0da-eb4c2fc38789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.584735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee461ac-6432-4d5a-a0da-eb4c2fc38789 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.585019 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee461ac-6432-4d5a-a0da-eb4c2fc38789 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.585287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee461ac-6432-4d5a-a0da-eb4c2fc38789 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.585972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee461ac-6432-4d5a-a0da-eb4c2fc38789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.588118 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee461ac-6432-4d5a-a0da-eb4c2fc38789 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.675700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 786/1593] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:41 2022] PUT /identity/v3/projects/3d04940b01354bcabce0a75683732b0d/users/3c5d9ba77968401da3c87d08b6b6027b/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:41.676458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f444e-7f3e-4232-9ac9-cc218acdd3a5 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.717561 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 808/1594] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:41 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:41.719165 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee461ac-6432-4d5a-a0da-eb4c2fc38789 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.752959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:41.754050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f444e-7f3e-4232-9ac9-cc218acdd3a5 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.796823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee461ac-6432-4d5a-a0da-eb4c2fc38789 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.801825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61ef5e-1abc-44e8-a5fc-27cd1ee7b1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.802497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61ef5e-1abc-44e8-a5fc-27cd1ee7b1ca admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.802777 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61ef5e-1abc-44e8-a5fc-27cd1ee7b1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.803047 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61ef5e-1abc-44e8-a5fc-27cd1ee7b1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.803876 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61ef5e-1abc-44e8-a5fc-27cd1ee7b1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.805475 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61ef5e-1abc-44e8-a5fc-27cd1ee7b1ca admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.806560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:41.817940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 809/1595] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:41.819207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61ef5e-1abc-44e8-a5fc-27cd1ee7b1ca admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.820938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75083c9-327c-43d1-9140-563d547f74c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.821741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75083c9-327c-43d1-9140-563d547f74c1 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.822119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75083c9-327c-43d1-9140-563d547f74c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.822594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75083c9-327c-43d1-9140-563d547f74c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.824194 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75083c9-327c-43d1-9140-563d547f74c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.826195 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75083c9-327c-43d1-9140-563d547f74c1 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.834545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61ef5e-1abc-44e8-a5fc-27cd1ee7b1ca admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.837525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 787/1596] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:41.839513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75083c9-327c-43d1-9140-563d547f74c1 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.841872 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa6e38-c314-4526-bb26-752f3fccc522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.842550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa6e38-c314-4526-bb26-752f3fccc522 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.842827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa6e38-c314-4526-bb26-752f3fccc522 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.843108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa6e38-c314-4526-bb26-752f3fccc522 admin admin] PATH_INFO: `/v3/projects/08db5f6552054a6c97ed644366d18b22/users/a1bc5bf55e9d452b88757ea4080165e0/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.851040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75083c9-327c-43d1-9140-563d547f74c1 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.857987 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f5355-1bfd-4649-be3c-a4d964e85292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.858945 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f5355-1bfd-4649-be3c-a4d964e85292 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.859392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f5355-1bfd-4649-be3c-a4d964e85292 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.859891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f5355-1bfd-4649-be3c-a4d964e85292 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.861100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f5355-1bfd-4649-be3c-a4d964e85292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.861626 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa6e38-c314-4526-bb26-752f3fccc522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08db5f6552054a6c97ed644366d18b22, user_id=a1bc5bf55e9d452b88757ea4080165e0, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.864152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f5355-1bfd-4649-be3c-a4d964e85292 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.864836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa6e38-c314-4526-bb26-752f3fccc522 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.876623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 788/1597] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:41.877670 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f5355-1bfd-4649-be3c-a4d964e85292 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.891227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f5355-1bfd-4649-be3c-a4d964e85292 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.895989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df81906-b667-46e4-9bd1-a6457245bf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.896606 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df81906-b667-46e4-9bd1-a6457245bf56 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.896885 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df81906-b667-46e4-9bd1-a6457245bf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.897159 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df81906-b667-46e4-9bd1-a6457245bf56 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.897857 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df81906-b667-46e4-9bd1-a6457245bf56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.899442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df81906-b667-46e4-9bd1-a6457245bf56 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:41.911218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 789/1598] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:41.912236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df81906-b667-46e4-9bd1-a6457245bf56 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:41.922763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df81906-b667-46e4-9bd1-a6457245bf56 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:41.927202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 810/1599] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:41 2022] PUT /identity/v3/projects/08db5f6552054a6c97ed644366d18b22/users/a1bc5bf55e9d452b88757ea4080165e0/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:41.928621 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1eed1e-82f7-4673-b2a5-5efe1ae3087d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.928905 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1eed1e-82f7-4673-b2a5-5efe1ae3087d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.929174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1eed1e-82f7-4673-b2a5-5efe1ae3087d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.965373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-1e1eed1e-82f7-4673-b2a5-5efe1ae3087d None None] MFA Rules not processed for user `3c5d9ba77968401da3c87d08b6b6027b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:41.966507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1e1eed1e-82f7-4673-b2a5-5efe1ae3087d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:41.994638 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49f043-c0ed-482a-a9cc-a11756093e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:41.996000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49f043-c0ed-482a-a9cc-a11756093e23 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:41.996287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49f043-c0ed-482a-a9cc-a11756093e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:41.996586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49f043-c0ed-482a-a9cc-a11756093e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:41.997388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49f043-c0ed-482a-a9cc-a11756093e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:41.999037 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49f043-c0ed-482a-a9cc-a11756093e23 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:42.010182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 790/1600] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:42.011729 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49f043-c0ed-482a-a9cc-a11756093e23 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:42.023143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49f043-c0ed-482a-a9cc-a11756093e23 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:42.029728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e707a7e-0e04-4044-9139-6b77492acad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:42.030370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e707a7e-0e04-4044-9139-6b77492acad6 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.030677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e707a7e-0e04-4044-9139-6b77492acad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.031202 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e707a7e-0e04-4044-9139-6b77492acad6 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.031689 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e707a7e-0e04-4044-9139-6b77492acad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:42.033671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e707a7e-0e04-4044-9139-6b77492acad6 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:42.046302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 811/1601] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:41 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:42.047136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1eed1e-82f7-4673-b2a5-5efe1ae3087d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:42.058851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1eed1e-82f7-4673-b2a5-5efe1ae3087d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:42.063628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e90329-3718-4dfa-8d44-eb50ee179edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:42.064318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e90329-3718-4dfa-8d44-eb50ee179edf admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.064592 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e90329-3718-4dfa-8d44-eb50ee179edf admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.064863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e90329-3718-4dfa-8d44-eb50ee179edf admin admin] PATH_INFO: `/v3/projects/2698a98c09a94ac0a09c50a6961fc431/users/d342d63f84d64871a35fd5aad3a254fa/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.082527 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e90329-3718-4dfa-8d44-eb50ee179edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2698a98c09a94ac0a09c50a6961fc431, user_id=d342d63f84d64871a35fd5aad3a254fa, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:42.085617 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e90329-3718-4dfa-8d44-eb50ee179edf admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:42.139044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 791/1602] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:42 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:42.140092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e707a7e-0e04-4044-9139-6b77492acad6 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:42.151015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e707a7e-0e04-4044-9139-6b77492acad6 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:42.155908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a530dc-f7fb-4b68-a597-f2eaec41ba66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:42.156633 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a530dc-f7fb-4b68-a597-f2eaec41ba66 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.156927 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a530dc-f7fb-4b68-a597-f2eaec41ba66 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.157210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a530dc-f7fb-4b68-a597-f2eaec41ba66 admin admin] PATH_INFO: `/v3/projects/242289f1dedc45ccbdd6aa8e19d4e414/users/d2738e8ef94a48bb9a0d6b1177598112/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.174896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a530dc-f7fb-4b68-a597-f2eaec41ba66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=242289f1dedc45ccbdd6aa8e19d4e414, user_id=d2738e8ef94a48bb9a0d6b1177598112, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:42.184284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a530dc-f7fb-4b68-a597-f2eaec41ba66 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:42.184613 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 812/1603] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:42 2022] PUT /identity/v3/projects/2698a98c09a94ac0a09c50a6961fc431/users/d342d63f84d64871a35fd5aad3a254fa/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:42.184613 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e90329-3718-4dfa-8d44-eb50ee179edf admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:42.254468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e90329-3718-4dfa-8d44-eb50ee179edf admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:42.259799 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332c5b1-13b0-4ceb-a8f2-843026244eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:42.260554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332c5b1-13b0-4ceb-a8f2-843026244eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.260960 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332c5b1-13b0-4ceb-a8f2-843026244eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.261348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332c5b1-13b0-4ceb-a8f2-843026244eb7 admin admin] PATH_INFO: `/v3/projects/bb18a5955f814887bcfa9650bcaa2ce9/users/c15d08b7eb014af3afc4d0a6e032f740/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.265454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:42.278259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332c5b1-13b0-4ceb-a8f2-843026244eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb18a5955f814887bcfa9650bcaa2ce9, user_id=c15d08b7eb014af3afc4d0a6e032f740, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:42.281134 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332c5b1-13b0-4ceb-a8f2-843026244eb7 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:42.296446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 792/1604] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:42 2022] PUT /identity/v3/projects/242289f1dedc45ccbdd6aa8e19d4e414/users/d2738e8ef94a48bb9a0d6b1177598112/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:42.301767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a530dc-f7fb-4b68-a597-f2eaec41ba66 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:42.379001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a530dc-f7fb-4b68-a597-f2eaec41ba66 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:42.385510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea6195-cf06-470b-8989-4c2d406c4458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:42.385510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea6195-cf06-470b-8989-4c2d406c4458 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.385510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea6195-cf06-470b-8989-4c2d406c4458 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.385510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea6195-cf06-470b-8989-4c2d406c4458 admin admin] PATH_INFO: `/v3/projects/dbe01616efc64d2e901f45662e7c3518/users/61b3efca1989483d960b9f92d655d39e/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.401327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea6195-cf06-470b-8989-4c2d406c4458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbe01616efc64d2e901f45662e7c3518, user_id=61b3efca1989483d960b9f92d655d39e, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:42.404083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea6195-cf06-470b-8989-4c2d406c4458 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:42.453468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 813/1605] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:42 2022] PUT /identity/v3/projects/bb18a5955f814887bcfa9650bcaa2ce9/users/c15d08b7eb014af3afc4d0a6e032f740/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:42.453468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8826456-9938-4869-bbb7-02c892311928 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.453468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8826456-9938-4869-bbb7-02c892311928 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.453468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8826456-9938-4869-bbb7-02c892311928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.462716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b8826456-9938-4869-bbb7-02c892311928 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:42.463963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b8826456-9938-4869-bbb7-02c892311928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:42.528455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 814/1606] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:42.529293 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8826456-9938-4869-bbb7-02c892311928 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:42.539167 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8826456-9938-4869-bbb7-02c892311928 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:42.544753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5afcad-6de0-477b-aa98-8113b3cce7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:42.545421 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5afcad-6de0-477b-aa98-8113b3cce7d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.545699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5afcad-6de0-477b-aa98-8113b3cce7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.545981 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5afcad-6de0-477b-aa98-8113b3cce7d7 admin admin] PATH_INFO: `/v3/projects/824168ef21a7473cbba25de87a9fdb48/users/3f85f81bc8db4e6aa6005e273cdecc35/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.550662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:42.555934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 793/1607] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:42 2022] PUT /identity/v3/projects/dbe01616efc64d2e901f45662e7c3518/users/61b3efca1989483d960b9f92d655d39e/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:42.564869 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea6195-cf06-470b-8989-4c2d406c4458 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:42.565039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5afcad-6de0-477b-aa98-8113b3cce7d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=824168ef21a7473cbba25de87a9fdb48, user_id=3f85f81bc8db4e6aa6005e273cdecc35, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:42.565554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5afcad-6de0-477b-aa98-8113b3cce7d7 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:42.628102 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea6195-cf06-470b-8989-4c2d406c4458 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:42.633236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74d866-c855-43b0-8206-d7ea852a0110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:42.633870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74d866-c855-43b0-8206-d7ea852a0110 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.634184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74d866-c855-43b0-8206-d7ea852a0110 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.634463 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74d866-c855-43b0-8206-d7ea852a0110 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.635145 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74d866-c855-43b0-8206-d7ea852a0110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:42.637187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74d866-c855-43b0-8206-d7ea852a0110 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:42.673497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 815/1608] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:42 2022] PUT /identity/v3/projects/824168ef21a7473cbba25de87a9fdb48/users/3f85f81bc8db4e6aa6005e273cdecc35/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:42.676129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5afcad-6de0-477b-aa98-8113b3cce7d7 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:42.739530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5afcad-6de0-477b-aa98-8113b3cce7d7 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:42.744278 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d964c-6850-47dd-898d-50ae086ba53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:42.744930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d964c-6850-47dd-898d-50ae086ba53c admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.745417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d964c-6850-47dd-898d-50ae086ba53c admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.745681 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d964c-6850-47dd-898d-50ae086ba53c admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.746386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d964c-6850-47dd-898d-50ae086ba53c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:42.747949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d964c-6850-47dd-898d-50ae086ba53c admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:42.758995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 816/1609] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:42.760543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648aaf0-2a72-4541-9c3b-3b4c7621930f None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.760966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648aaf0-2a72-4541-9c3b-3b4c7621930f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.761217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648aaf0-2a72-4541-9c3b-3b4c7621930f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.796500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-9648aaf0-2a72-4541-9c3b-3b4c7621930f None None] MFA Rules not processed for user `d342d63f84d64871a35fd5aad3a254fa`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:42.797796 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9648aaf0-2a72-4541-9c3b-3b4c7621930f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:42.810039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 794/1610] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:42 2022] POST /identity/v3/projects => generated 355 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:42.810039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14b73d-32c7-432f-bd0f-6a7ab3714658 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.810039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14b73d-32c7-432f-bd0f-6a7ab3714658 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.810039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14b73d-32c7-432f-bd0f-6a7ab3714658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.843022 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-1c14b73d-32c7-432f-bd0f-6a7ab3714658 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:42.844594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1c14b73d-32c7-432f-bd0f-6a7ab3714658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:42.872382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:42.904017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 817/1611] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:42.904017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973729af-3aa7-46fb-91c8-bbe4b27d0b88 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.904017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973729af-3aa7-46fb-91c8-bbe4b27d0b88 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.904017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973729af-3aa7-46fb-91c8-bbe4b27d0b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.924599 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-973729af-3aa7-46fb-91c8-bbe4b27d0b88 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:42.925885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-973729af-3aa7-46fb-91c8-bbe4b27d0b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:42.934135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 795/1612] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:42.935744 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14b73d-32c7-432f-bd0f-6a7ab3714658 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:42.937389 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1c14b73d-32c7-432f-bd0f-6a7ab3714658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:42.939102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 818/1613] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:42.940777 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727a02d-94e8-4269-a6e1-dd858ee90a11 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.941138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727a02d-94e8-4269-a6e1-dd858ee90a11 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.941526 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727a02d-94e8-4269-a6e1-dd858ee90a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:42.980194 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-4727a02d-94e8-4269-a6e1-dd858ee90a11 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:42.981911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4727a02d-94e8-4269-a6e1-dd858ee90a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:42.995306 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 819/1614] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:42.995306 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493efce-98cc-4ebf-b661-258d20e30bf1 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:42.995306 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493efce-98cc-4ebf-b661-258d20e30bf1 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:42.995306 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493efce-98cc-4ebf-b661-258d20e30bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:43.021178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14b73d-32c7-432f-bd0f-6a7ab3714658 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:43.028526 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772da7b-6bc0-45f4-9085-69071f5e6a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:43.029177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772da7b-6bc0-45f4-9085-69071f5e6a88 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:43.029465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772da7b-6bc0-45f4-9085-69071f5e6a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:43.029748 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772da7b-6bc0-45f4-9085-69071f5e6a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:43.030519 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-5493efce-98cc-4ebf-b661-258d20e30bf1 None None] MFA Rules not processed for user `3f85f81bc8db4e6aa6005e273cdecc35`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:43.031791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5493efce-98cc-4ebf-b661-258d20e30bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:43.040118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772da7b-6bc0-45f4-9085-69071f5e6a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:43.041637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772da7b-6bc0-45f4-9085-69071f5e6a88 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:43.055397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 796/1615] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:43.056622 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772da7b-6bc0-45f4-9085-69071f5e6a88 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:43.067030 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772da7b-6bc0-45f4-9085-69071f5e6a88 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:43.072931 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca6b49-8586-4ad9-85fd-8ff2de049138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:43.076265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca6b49-8586-4ad9-85fd-8ff2de049138 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:43.076265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca6b49-8586-4ad9-85fd-8ff2de049138 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:43.076265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca6b49-8586-4ad9-85fd-8ff2de049138 admin admin] PATH_INFO: `/v3/projects/7066fcb933a84548b8b2684ba5e7c98f/users/39939f217b3544dfa62cd94944a17052/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:43.102068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca6b49-8586-4ad9-85fd-8ff2de049138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7066fcb933a84548b8b2684ba5e7c98f, user_id=39939f217b3544dfa62cd94944a17052, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:43.105001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca6b49-8586-4ad9-85fd-8ff2de049138 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:43.119828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 820/1616] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:43.120805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493efce-98cc-4ebf-b661-258d20e30bf1 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:43.132576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493efce-98cc-4ebf-b661-258d20e30bf1 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:43.138817 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a2361-3c50-4b34-8cb1-66275459628f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:43.139518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a2361-3c50-4b34-8cb1-66275459628f admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:43.139846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a2361-3c50-4b34-8cb1-66275459628f admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:43.140168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a2361-3c50-4b34-8cb1-66275459628f admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:43.140920 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a2361-3c50-4b34-8cb1-66275459628f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:43.143011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a2361-3c50-4b34-8cb1-66275459628f admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:43.285998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 797/1617] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:43 2022] PUT /identity/v3/projects/7066fcb933a84548b8b2684ba5e7c98f/users/39939f217b3544dfa62cd94944a17052/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:43.287360 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba219e-0970-4dbf-b99b-c0ab7d43391f None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:43.287695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba219e-0970-4dbf-b99b-c0ab7d43391f None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:43.288130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba219e-0970-4dbf-b99b-c0ab7d43391f None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:43.289545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 798/1618] 10.222.0.94 () {54 vars in 962 bytes} [Tue Jul 26 15:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:04:43.290485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba219e-0970-4dbf-b99b-c0ab7d43391f None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:43.298609 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 821/1619] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:43 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:43.298609 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a2361-3c50-4b34-8cb1-66275459628f admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:43.302141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:43.310430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a2361-3c50-4b34-8cb1-66275459628f admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:43.369538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba219e-0970-4dbf-b99b-c0ab7d43391f None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:43.373019 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c15b4d-7614-42e0-83dd-6ba3150648ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:43.373658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c15b4d-7614-42e0-83dd-6ba3150648ad service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:43.373965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c15b4d-7614-42e0-83dd-6ba3150648ad service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:43.374468 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66a6fc-34a2-4293-bf50-9becefd8cf5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:43.374683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c15b4d-7614-42e0-83dd-6ba3150648ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:43.375110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66a6fc-34a2-4293-bf50-9becefd8cf5e service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:43.376005 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66a6fc-34a2-4293-bf50-9becefd8cf5e service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:43.376289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66a6fc-34a2-4293-bf50-9becefd8cf5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:43.470796 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:43.473712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c15b4d-7614-42e0-83dd-6ba3150648ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:43.475254 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c15b4d-7614-42e0-83dd-6ba3150648ad service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:43.475387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf66a6fc-34a2-4293-bf50-9becefd8cf5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:43.478218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf66a6fc-34a2-4293-bf50-9becefd8cf5e service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:43.487529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 822/1620] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:43.488286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c15b4d-7614-42e0-83dd-6ba3150648ad service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:43.498253 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c15b4d-7614-42e0-83dd-6ba3150648ad service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:43.498701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 799/1621] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:43.502044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66a6fc-34a2-4293-bf50-9becefd8cf5e service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:43.503433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ef895-bd2f-4676-9011-06c57e03c7fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:43.504128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ef895-bd2f-4676-9011-06c57e03c7fa service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:43.504501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ef895-bd2f-4676-9011-06c57e03c7fa service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:43.504655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ef895-bd2f-4676-9011-06c57e03c7fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:43.513386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66a6fc-34a2-4293-bf50-9becefd8cf5e service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:43.517754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ef895-bd2f-4676-9011-06c57e03c7fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:43.519321 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ef895-bd2f-4676-9011-06c57e03c7fa service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:43.526207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05361182-e1ef-4662-b016-bb809518b091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:43.528338 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361182-e1ef-4662-b016-bb809518b091 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:43.528681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361182-e1ef-4662-b016-bb809518b091 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:43.529015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361182-e1ef-4662-b016-bb809518b091 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:43.529778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05361182-e1ef-4662-b016-bb809518b091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:43.532033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05361182-e1ef-4662-b016-bb809518b091 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:43.533743 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 823/1622] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:43.535571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ef895-bd2f-4676-9011-06c57e03c7fa service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:43.546582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ef895-bd2f-4676-9011-06c57e03c7fa service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:43.555739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd266e-1da8-4f73-84c6-b51aa7e1cfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:43.556094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd266e-1da8-4f73-84c6-b51aa7e1cfce admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:43.556374 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd266e-1da8-4f73-84c6-b51aa7e1cfce admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:43.556727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd266e-1da8-4f73-84c6-b51aa7e1cfce admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:43.557417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd266e-1da8-4f73-84c6-b51aa7e1cfce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:43.560057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd266e-1da8-4f73-84c6-b51aa7e1cfce admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:43.575693 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 824/1623] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:43.575693 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6603be-5a28-4c31-b973-9816c7ebfe73 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:43.575693 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6603be-5a28-4c31-b973-9816c7ebfe73 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:43.575693 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6603be-5a28-4c31-b973-9816c7ebfe73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:43.620458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-9f6603be-5a28-4c31-b973-9816c7ebfe73 None None] MFA Rules not processed for user `6a4278fe995e432d88f4346d6f54218b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:43.622029 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9f6603be-5a28-4c31-b973-9816c7ebfe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:43.682922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 800/1624] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:43 2022] POST /identity/v3/projects => generated 365 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:43.682922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05361182-e1ef-4662-b016-bb809518b091 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:43.703180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 825/1625] 10.222.0.94 () {58 vars in 1059 bytes} [Tue Jul 26 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:43.703561 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6603be-5a28-4c31-b973-9816c7ebfe73 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:43.717947 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6603be-5a28-4c31-b973-9816c7ebfe73 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:43.771267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05361182-e1ef-4662-b016-bb809518b091 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:43.778603 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2e0a1-2045-45cd-ba22-00a83ba9c986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:43.779318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2e0a1-2045-45cd-ba22-00a83ba9c986 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.005561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2e0a1-2045-45cd-ba22-00a83ba9c986 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.005561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2e0a1-2045-45cd-ba22-00a83ba9c986 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.005561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2e0a1-2045-45cd-ba22-00a83ba9c986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:44.005561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2e0a1-2045-45cd-ba22-00a83ba9c986 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:44.005561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 801/1626] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:44.005561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2e0a1-2045-45cd-ba22-00a83ba9c986 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:44.008376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dd385-e287-46b6-a861-20bc3d346b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:44.009146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2e0a1-2045-45cd-ba22-00a83ba9c986 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:44.010243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dd385-e287-46b6-a861-20bc3d346b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.010243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dd385-e287-46b6-a861-20bc3d346b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.010243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dd385-e287-46b6-a861-20bc3d346b2a admin admin] PATH_INFO: `/v3/projects/7066fcb933a84548b8b2684ba5e7c98f/users/39939f217b3544dfa62cd94944a17052/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.010243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51dd385-e287-46b6-a861-20bc3d346b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7066fcb933a84548b8b2684ba5e7c98f, user_id=39939f217b3544dfa62cd94944a17052, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:44.011609 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc3334-fd90-4c03-8e88-800e28dc50cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:44.011798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51dd385-e287-46b6-a861-20bc3d346b2a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:44.011977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc3334-fd90-4c03-8e88-800e28dc50cf admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.011977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc3334-fd90-4c03-8e88-800e28dc50cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.011977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc3334-fd90-4c03-8e88-800e28dc50cf admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.011977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc3334-fd90-4c03-8e88-800e28dc50cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:44.011977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc3334-fd90-4c03-8e88-800e28dc50cf admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:44.013353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 826/1627] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:43 2022] PUT /identity/v3/projects/7066fcb933a84548b8b2684ba5e7c98f/users/39939f217b3544dfa62cd94944a17052/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:44.013353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dd385-e287-46b6-a861-20bc3d346b2a admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:44.013353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:44.013353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dd385-e287-46b6-a861-20bc3d346b2a admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:44.016070 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17b9bc-0833-4fba-9718-e0def0a82b90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:44.016070 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17b9bc-0833-4fba-9718-e0def0a82b90 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.016070 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17b9bc-0833-4fba-9718-e0def0a82b90 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.016070 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17b9bc-0833-4fba-9718-e0def0a82b90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.017207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 802/1628] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:43 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:44.017207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc3334-fd90-4c03-8e88-800e28dc50cf admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:44.017207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc3334-fd90-4c03-8e88-800e28dc50cf admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:44.059526 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:44.068643 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17b9bc-0833-4fba-9718-e0def0a82b90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:44.071774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17b9bc-0833-4fba-9718-e0def0a82b90 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:44.093615 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 827/1629] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:44.096443 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749274f6-9b3b-4ffc-b72b-f66a4d08752d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.097122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749274f6-9b3b-4ffc-b72b-f66a4d08752d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.097606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749274f6-9b3b-4ffc-b72b-f66a4d08752d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.147750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ecd5e-03b2-4c6c-80b1-01e02b758daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:44.149669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ecd5e-03b2-4c6c-80b1-01e02b758daf admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.150167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ecd5e-03b2-4c6c-80b1-01e02b758daf admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.150694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ecd5e-03b2-4c6c-80b1-01e02b758daf admin admin] PATH_INFO: `/v3/projects/75fa510331d3467d96dd647e7d65cf5e/users/16601214f4ec49c08798a1e30f65c804/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.176759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ecd5e-03b2-4c6c-80b1-01e02b758daf admin admin] RBAC: Authorizing `identity:create_grant(project_id=75fa510331d3467d96dd647e7d65cf5e, user_id=16601214f4ec49c08798a1e30f65c804, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:44.180035 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ecd5e-03b2-4c6c-80b1-01e02b758daf admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:44.182915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-749274f6-9b3b-4ffc-b72b-f66a4d08752d None None] MFA Rules not processed for user `39939f217b3544dfa62cd94944a17052`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:44.184885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-749274f6-9b3b-4ffc-b72b-f66a4d08752d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:44.304673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 828/1630] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:44.305022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749274f6-9b3b-4ffc-b72b-f66a4d08752d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:44.319956 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749274f6-9b3b-4ffc-b72b-f66a4d08752d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:44.334835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66847218-afb0-4706-b9f0-be3da5108ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:44.336561 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66847218-afb0-4706-b9f0-be3da5108ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.336927 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66847218-afb0-4706-b9f0-be3da5108ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.337284 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66847218-afb0-4706-b9f0-be3da5108ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.338333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66847218-afb0-4706-b9f0-be3da5108ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:44.342005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66847218-afb0-4706-b9f0-be3da5108ba0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:44.357614 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 803/1631] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:43 2022] PUT /identity/v3/projects/75fa510331d3467d96dd647e7d65cf5e/users/16601214f4ec49c08798a1e30f65c804/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:44.358810 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ecd5e-03b2-4c6c-80b1-01e02b758daf admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:44.386836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 829/1632] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:44.387798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66847218-afb0-4706-b9f0-be3da5108ba0 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:44.440253 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66847218-afb0-4706-b9f0-be3da5108ba0 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:44.540514 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ecd5e-03b2-4c6c-80b1-01e02b758daf admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:44.557347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dbb086-920f-4f0a-a924-6b4a158756b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:44.558063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbb086-920f-4f0a-a924-6b4a158756b1 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.558407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbb086-920f-4f0a-a924-6b4a158756b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.558690 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbb086-920f-4f0a-a924-6b4a158756b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.576561 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:44.585625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd58d64-6ca7-4e7a-b759-eeedcf83304e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:44.586445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd58d64-6ca7-4e7a-b759-eeedcf83304e admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.586734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd58d64-6ca7-4e7a-b759-eeedcf83304e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.587003 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd58d64-6ca7-4e7a-b759-eeedcf83304e admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.588593 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd58d64-6ca7-4e7a-b759-eeedcf83304e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:44.591325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd58d64-6ca7-4e7a-b759-eeedcf83304e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:44.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 830/1633] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:44.610720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd58d64-6ca7-4e7a-b759-eeedcf83304e admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:44.621974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd58d64-6ca7-4e7a-b759-eeedcf83304e admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:44.630069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5d4e9-245e-4dcd-8ca4-6929bf77949d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:44.631297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5d4e9-245e-4dcd-8ca4-6929bf77949d admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.631297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5d4e9-245e-4dcd-8ca4-6929bf77949d admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.631713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5d4e9-245e-4dcd-8ca4-6929bf77949d admin admin] PATH_INFO: `/v3/projects/83add6533a98421393be43d7b04b0cb3/users/58fdd473348c4baa9744e5ef480a0bca/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.651528 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5d4e9-245e-4dcd-8ca4-6929bf77949d admin admin] RBAC: Authorizing `identity:create_grant(project_id=83add6533a98421393be43d7b04b0cb3, user_id=58fdd473348c4baa9744e5ef480a0bca, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:44.654734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5d4e9-245e-4dcd-8ca4-6929bf77949d admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:44.676081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dbb086-920f-4f0a-a924-6b4a158756b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:44.677658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dbb086-920f-4f0a-a924-6b4a158756b1 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:44.691050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 804/1634] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:44.692532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43cf1b-fe4d-44bc-819f-76479c815c05 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.692903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43cf1b-fe4d-44bc-819f-76479c815c05 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.693281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43cf1b-fe4d-44bc-819f-76479c815c05 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.694735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 805/1635] 10.222.0.94 () {54 vars in 912 bytes} [Tue Jul 26 15:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:04:44.700955 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43cf1b-fe4d-44bc-819f-76479c815c05 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:44.717366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43cf1b-fe4d-44bc-819f-76479c815c05 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:44.723259 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b440b-63d9-4b82-a7a0-a316fba826b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:44.732413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b440b-63d9-4b82-a7a0-a316fba826b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.733664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b440b-63d9-4b82-a7a0-a316fba826b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.734459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b440b-63d9-4b82-a7a0-a316fba826b8 admin admin] PATH_INFO: `/v3/projects/75fa510331d3467d96dd647e7d65cf5e/users/16601214f4ec49c08798a1e30f65c804/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.737887 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b440b-63d9-4b82-a7a0-a316fba826b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75fa510331d3467d96dd647e7d65cf5e, user_id=16601214f4ec49c08798a1e30f65c804, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:44.742876 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b440b-63d9-4b82-a7a0-a316fba826b8 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:44.744424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 831/1636] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:44 2022] PUT /identity/v3/projects/83add6533a98421393be43d7b04b0cb3/users/58fdd473348c4baa9744e5ef480a0bca/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:44.746789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90df742-2aa7-45d0-a7b4-865b9bca9fdb None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.746789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90df742-2aa7-45d0-a7b4-865b9bca9fdb None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.747006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90df742-2aa7-45d0-a7b4-865b9bca9fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.761479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 806/1637] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:44 2022] PUT /identity/v3/projects/75fa510331d3467d96dd647e7d65cf5e/users/16601214f4ec49c08798a1e30f65c804/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:44.763046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b440b-63d9-4b82-a7a0-a316fba826b8 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:44.792969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-e90df742-2aa7-45d0-a7b4-865b9bca9fdb None None] MFA Rules not processed for user `6a4278fe995e432d88f4346d6f54218b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:44.796626 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e90df742-2aa7-45d0-a7b4-865b9bca9fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:44.831699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:44.870506 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b440b-63d9-4b82-a7a0-a316fba826b8 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:44.880177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa192708-01ab-4d1d-bb78-8722e753799f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:44.880975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa192708-01ab-4d1d-bb78-8722e753799f admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.881368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa192708-01ab-4d1d-bb78-8722e753799f admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.881756 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa192708-01ab-4d1d-bb78-8722e753799f admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.887745 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa192708-01ab-4d1d-bb78-8722e753799f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:44.887745 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa192708-01ab-4d1d-bb78-8722e753799f admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:44.902634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 807/1638] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:44.902634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6680115-8363-4f1a-bb0d-3575eb6c03e4 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:44.902634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6680115-8363-4f1a-bb0d-3575eb6c03e4 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:44.902634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6680115-8363-4f1a-bb0d-3575eb6c03e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:44.953403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-f6680115-8363-4f1a-bb0d-3575eb6c03e4 None None] MFA Rules not processed for user `16601214f4ec49c08798a1e30f65c804`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:44.954752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f6680115-8363-4f1a-bb0d-3575eb6c03e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:44.979304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 832/1639] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 15:04:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:44.981201 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90df742-2aa7-45d0-a7b4-865b9bca9fdb None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:44.993684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90df742-2aa7-45d0-a7b4-865b9bca9fdb None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:45.000800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0acf4-21ab-4766-8a13-42645427890c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:45.001683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0acf4-21ab-4766-8a13-42645427890c admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:45.002499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0acf4-21ab-4766-8a13-42645427890c admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:45.003024 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0acf4-21ab-4766-8a13-42645427890c admin admin] PATH_INFO: `/v3/projects/83add6533a98421393be43d7b04b0cb3/users/58fdd473348c4baa9744e5ef480a0bca/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:45.005723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0acf4-21ab-4766-8a13-42645427890c admin admin] RBAC: Authorizing `identity:create_grant(project_id=83add6533a98421393be43d7b04b0cb3, user_id=58fdd473348c4baa9744e5ef480a0bca, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:45.009726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0acf4-21ab-4766-8a13-42645427890c admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:45.022901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 833/1640] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:44 2022] PUT /identity/v3/projects/83add6533a98421393be43d7b04b0cb3/users/58fdd473348c4baa9744e5ef480a0bca/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:45.031993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0acf4-21ab-4766-8a13-42645427890c admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:45.082295 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 808/1641] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:44 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:45.082295 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed90dcb-79f7-4728-9578-2ef5e571fa30 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:45.082295 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed90dcb-79f7-4728-9578-2ef5e571fa30 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:45.082295 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed90dcb-79f7-4728-9578-2ef5e571fa30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:45.115359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0acf4-21ab-4766-8a13-42645427890c admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:45.116459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-bed90dcb-79f7-4728-9578-2ef5e571fa30 None None] MFA Rules not processed for user `58fdd473348c4baa9744e5ef480a0bca`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:45.117864 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-bed90dcb-79f7-4728-9578-2ef5e571fa30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:45.121258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e608190-40e5-44a3-9d6d-2e8fa21bcf3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:45.122101 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e608190-40e5-44a3-9d6d-2e8fa21bcf3c service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:45.122572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e608190-40e5-44a3-9d6d-2e8fa21bcf3c service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:45.122984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e608190-40e5-44a3-9d6d-2e8fa21bcf3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:45.247738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 809/1642] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:45 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:45.248650 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:45.248942 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed90dcb-79f7-4728-9578-2ef5e571fa30 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:45.270036 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed90dcb-79f7-4728-9578-2ef5e571fa30 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:45.279294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372dcdc-da23-4aef-8f4b-9d17252c18ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:45.279948 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e608190-40e5-44a3-9d6d-2e8fa21bcf3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:45.280208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372dcdc-da23-4aef-8f4b-9d17252c18ee service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:45.280331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372dcdc-da23-4aef-8f4b-9d17252c18ee service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:45.280512 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372dcdc-da23-4aef-8f4b-9d17252c18ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:45.282935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e608190-40e5-44a3-9d6d-2e8fa21bcf3c service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:45.310975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 834/1643] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:45.396343 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372dcdc-da23-4aef-8f4b-9d17252c18ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:45.397894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372dcdc-da23-4aef-8f4b-9d17252c18ee service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:45.424395 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 810/1644] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:45 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:45.896289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:46.148189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e608190-40e5-44a3-9d6d-2e8fa21bcf3c service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:46.160311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e608190-40e5-44a3-9d6d-2e8fa21bcf3c service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:46.166009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0fdf7-3f04-41eb-8d77-31e070c2de80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:46.169516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0fdf7-3f04-41eb-8d77-31e070c2de80 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:46.169801 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0fdf7-3f04-41eb-8d77-31e070c2de80 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:46.170116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0fdf7-3f04-41eb-8d77-31e070c2de80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:46.261584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0fdf7-3f04-41eb-8d77-31e070c2de80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:46.266622 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0fdf7-3f04-41eb-8d77-31e070c2de80 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:46.288182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 835/1645] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:46 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:46.492061 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372dcdc-da23-4aef-8f4b-9d17252c18ee service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:46.502244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372dcdc-da23-4aef-8f4b-9d17252c18ee service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:46.507214 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386242a-3443-4839-a5ea-f702f1cfb77d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:46.515979 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386242a-3443-4839-a5ea-f702f1cfb77d service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:46.515979 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386242a-3443-4839-a5ea-f702f1cfb77d service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:46.515979 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386242a-3443-4839-a5ea-f702f1cfb77d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:46.523790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386242a-3443-4839-a5ea-f702f1cfb77d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:46.525565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386242a-3443-4839-a5ea-f702f1cfb77d service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:46.540631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 811/1646] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:47.281771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0fdf7-3f04-41eb-8d77-31e070c2de80 service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:47.302199 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0fdf7-3f04-41eb-8d77-31e070c2de80 service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:47.310355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9047a05-0e96-46ef-af96-0afb8c28f9b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:47.311784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9047a05-0e96-46ef-af96-0afb8c28f9b5 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:47.312524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9047a05-0e96-46ef-af96-0afb8c28f9b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:47.313185 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9047a05-0e96-46ef-af96-0afb8c28f9b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:47.495494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9047a05-0e96-46ef-af96-0afb8c28f9b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:47.497809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9047a05-0e96-46ef-af96-0afb8c28f9b5 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:47.510023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 836/1647] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:47 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:47.629948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c56d11-a4b8-4e2d-a02b-620e4e4649c7 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:47.630979 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c56d11-a4b8-4e2d-a02b-620e4e4649c7 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:47.631412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c56d11-a4b8-4e2d-a02b-620e4e4649c7 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:47.633319 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 812/1648] 10.222.0.94 () {54 vars in 962 bytes} [Tue Jul 26 15:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:04:47.642179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d197d95-bb21-47da-8c78-d250bc5dc8c7 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:47.642896 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d197d95-bb21-47da-8c78-d250bc5dc8c7 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:47.643496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d197d95-bb21-47da-8c78-d250bc5dc8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:47.689793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-9d197d95-bb21-47da-8c78-d250bc5dc8c7 None None] MFA Rules not processed for user `6a4278fe995e432d88f4346d6f54218b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:47.691723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9d197d95-bb21-47da-8c78-d250bc5dc8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:47.710657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5687d-be04-4486-a434-2d525799b8e9 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:47.711180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5687d-be04-4486-a434-2d525799b8e9 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:47.711667 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5687d-be04-4486-a434-2d525799b8e9 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:47.713445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 813/1649] 10.222.0.94 () {54 vars in 912 bytes} [Tue Jul 26 15:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 15:04:47.720561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0ab30-6fb6-483e-932e-756c93820871 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:47.721126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0ab30-6fb6-483e-932e-756c93820871 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:47.721607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0ab30-6fb6-483e-932e-756c93820871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:47.736010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 837/1650] 10.222.0.94 () {58 vars in 1059 bytes} [Tue Jul 26 15:04:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:04:47.764793 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-0cc0ab30-6fb6-483e-932e-756c93820871 None None] MFA Rules not processed for user `6a4278fe995e432d88f4346d6f54218b`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:47.766416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0cc0ab30-6fb6-483e-932e-756c93820871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:47.784615 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 814/1651] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 15:04:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:04:48.014386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d197d95-bb21-47da-8c78-d250bc5dc8c7 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:48.029713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d197d95-bb21-47da-8c78-d250bc5dc8c7 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:48.034613 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3eac03-a356-40c4-a4e8-eefcb8adffd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:48.035543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eac03-a356-40c4-a4e8-eefcb8adffd7 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:48.035899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eac03-a356-40c4-a4e8-eefcb8adffd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:48.036279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eac03-a356-40c4-a4e8-eefcb8adffd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:48.067404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3eac03-a356-40c4-a4e8-eefcb8adffd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:48.075778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3eac03-a356-40c4-a4e8-eefcb8adffd7 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:48.095875 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 838/1652] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:49.487201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0ab30-6fb6-483e-932e-756c93820871 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:49.715789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0ab30-6fb6-483e-932e-756c93820871 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:49.734531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:49.735592 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:49.736140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:49.736625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] PATH_INFO: `/v3/users/3f85f81bc8db4e6aa6005e273cdecc35` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:49.746380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f85f81bc8db4e6aa6005e273cdecc35)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:49.747922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: WARNING py.warnings [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:04:49.747922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: warnings.warn(msg) Jul 26 15:04:49.747922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: Jul 26 15:04:49.749836 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:50.137606 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f85f81bc8db4e6aa6005e273cdecc35'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:50.141003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f85f81bc8db4e6aa6005e273cdecc35'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:50.143905 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f85f81bc8db4e6aa6005e273cdecc35'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:50.192678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f85f81bc8db4e6aa6005e273cdecc35'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:50.249782 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f85f81bc8db4e6aa6005e273cdecc35'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:50.377067 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 815/1653] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:04:49 2022] DELETE /identity/v3/users/3f85f81bc8db4e6aa6005e273cdecc35 => generated 0 bytes in 890 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:50.382179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032b0e7-79e3-4cfd-8180-581b0f519eac None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:50.382784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032b0e7-79e3-4cfd-8180-581b0f519eac None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:50.382784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032b0e7-79e3-4cfd-8180-581b0f519eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:50.430073 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7032b0e7-79e3-4cfd-8180-581b0f519eac None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:50.431251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7032b0e7-79e3-4cfd-8180-581b0f519eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:50.552919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:50.577677 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 839/1654] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:50.621632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:50.624798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:50.737404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:50.780227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa4637-b767-45c7-8abd-0abb865565d0 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:50.785762 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6ed79-2096-43cc-a33d-188a153e0362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:50.786920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6ed79-2096-43cc-a33d-188a153e0362 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:50.787699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6ed79-2096-43cc-a33d-188a153e0362 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:50.788211 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6ed79-2096-43cc-a33d-188a153e0362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:50.801490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6ed79-2096-43cc-a33d-188a153e0362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:50.803788 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6ed79-2096-43cc-a33d-188a153e0362 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:50.825376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 816/1655] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:04:52.538383 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032b0e7-79e3-4cfd-8180-581b0f519eac None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:52.539908 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7032b0e7-79e3-4cfd-8180-581b0f519eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:52.564863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032b0e7-79e3-4cfd-8180-581b0f519eac None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:52.570719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:52.571962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:52.591410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:52.591410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] PATH_INFO: `/v3/projects/824168ef21a7473cbba25de87a9fdb48` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:52.591410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=824168ef21a7473cbba25de87a9fdb48)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:52.591410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: WARNING py.warnings [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 15:04:52.591410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: warnings.warn(msg) Jul 26 15:04:52.591410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: Jul 26 15:04:52.591410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:52.834794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '824168ef21a7473cbba25de87a9fdb48', 'request_id': 'req-7af493a0-b315-4a2d-a099-db4791d927a3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:52.848917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '824168ef21a7473cbba25de87a9fdb48', 'request_id': 'req-7af493a0-b315-4a2d-a099-db4791d927a3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:52.960267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '824168ef21a7473cbba25de87a9fdb48', 'request_id': 'req-7af493a0-b315-4a2d-a099-db4791d927a3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:52.998483 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] The token cache is being invalidate because project 824168ef21a7473cbba25de87a9fdb48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:04:52.999926 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:53.001208 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:53.009943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 840/1656] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:04:52 2022] DELETE /identity/v3/projects/824168ef21a7473cbba25de87a9fdb48 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:53.010125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6ed79-2096-43cc-a33d-188a153e0362 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:53.031636 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6ed79-2096-43cc-a33d-188a153e0362 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:53.172177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2edae6-857a-4537-815d-79fbbbe36d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:53.172880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2edae6-857a-4537-815d-79fbbbe36d71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:53.173172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2edae6-857a-4537-815d-79fbbbe36d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:53.173985 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2edae6-857a-4537-815d-79fbbbe36d71 admin admin] PATH_INFO: `/v3/users/58fdd473348c4baa9744e5ef480a0bca` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:53.175380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2edae6-857a-4537-815d-79fbbbe36d71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58fdd473348c4baa9744e5ef480a0bca)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:53.177401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2edae6-857a-4537-815d-79fbbbe36d71 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:53.496645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4b2edae6-857a-4537-815d-79fbbbe36d71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58fdd473348c4baa9744e5ef480a0bca'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:53.498739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4b2edae6-857a-4537-815d-79fbbbe36d71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58fdd473348c4baa9744e5ef480a0bca'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:53.502236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4b2edae6-857a-4537-815d-79fbbbe36d71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58fdd473348c4baa9744e5ef480a0bca'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:53.518992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4b2edae6-857a-4537-815d-79fbbbe36d71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58fdd473348c4baa9744e5ef480a0bca'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:53.536033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4b2edae6-857a-4537-815d-79fbbbe36d71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58fdd473348c4baa9744e5ef480a0bca'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:53.735174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 817/1657] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:04:53 2022] DELETE /identity/v3/users/58fdd473348c4baa9744e5ef480a0bca => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:56.570584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:56.572207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:56.627624 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af493a0-b315-4a2d-a099-db4791d927a3 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:56.795492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74aca7f-114a-48e6-8c18-3b34075d617f None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:56.796057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74aca7f-114a-48e6-8c18-3b34075d617f None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:56.796485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74aca7f-114a-48e6-8c18-3b34075d617f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:56.799828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7433d8c1-1474-4b76-8217-b697c6b111ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:56.801231 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433d8c1-1474-4b76-8217-b697c6b111ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:56.801553 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433d8c1-1474-4b76-8217-b697c6b111ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:56.801781 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433d8c1-1474-4b76-8217-b697c6b111ad admin admin] PATH_INFO: `/v3/projects/83add6533a98421393be43d7b04b0cb3` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:56.804260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7433d8c1-1474-4b76-8217-b697c6b111ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=83add6533a98421393be43d7b04b0cb3)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:56.808543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7433d8c1-1474-4b76-8217-b697c6b111ad admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:56.937575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-e74aca7f-114a-48e6-8c18-3b34075d617f None None] MFA Rules not processed for user `d2738e8ef94a48bb9a0d6b1177598112`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:56.939388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e74aca7f-114a-48e6-8c18-3b34075d617f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:57.039986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7433d8c1-1474-4b76-8217-b697c6b111ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83add6533a98421393be43d7b04b0cb3', 'request_id': 'req-7433d8c1-1474-4b76-8217-b697c6b111ad'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:57.054180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7433d8c1-1474-4b76-8217-b697c6b111ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83add6533a98421393be43d7b04b0cb3', 'request_id': 'req-7433d8c1-1474-4b76-8217-b697c6b111ad'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:57.113698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:57.159247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 818/1658] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:04:57.177393 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74aca7f-114a-48e6-8c18-3b34075d617f None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:57.179188 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e74aca7f-114a-48e6-8c18-3b34075d617f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:57.183773 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7433d8c1-1474-4b76-8217-b697c6b111ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83add6533a98421393be43d7b04b0cb3', 'request_id': 'req-7433d8c1-1474-4b76-8217-b697c6b111ad'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:57.206642 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7433d8c1-1474-4b76-8217-b697c6b111ad admin admin] The token cache is being invalidate because project 83add6533a98421393be43d7b04b0cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:04:57.207178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7433d8c1-1474-4b76-8217-b697c6b111ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:57.208106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7433d8c1-1474-4b76-8217-b697c6b111ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:04:57.215962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 841/1659] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:04:56 2022] DELETE /identity/v3/projects/83add6533a98421393be43d7b04b0cb3 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:57.217695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ea795-f1f4-4cc3-a971-db861e45921a None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:57.218144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ea795-f1f4-4cc3-a971-db861e45921a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:57.218558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ea795-f1f4-4cc3-a971-db861e45921a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:57.264524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-5a0ea795-f1f4-4cc3-a971-db861e45921a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:57.266072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5a0ea795-f1f4-4cc3-a971-db861e45921a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:57.289280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:57.445129 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74aca7f-114a-48e6-8c18-3b34075d617f None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:57.446387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 842/1660] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:57.457674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3cd4f-173e-4881-9c32-0679f01f131e None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:57.458914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3cd4f-173e-4881-9c32-0679f01f131e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:57.459934 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3cd4f-173e-4881-9c32-0679f01f131e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:57.528854 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-20c3cd4f-173e-4881-9c32-0679f01f131e None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:57.530176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-20c3cd4f-173e-4881-9c32-0679f01f131e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:57.569643 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 843/1661] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:04:57.570750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3cd4f-173e-4881-9c32-0679f01f131e None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:57.592813 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3cd4f-173e-4881-9c32-0679f01f131e None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:57.614515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb22175-23ec-49e6-8e03-c5df04f52cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:57.615147 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb22175-23ec-49e6-8e03-c5df04f52cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:57.615566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb22175-23ec-49e6-8e03-c5df04f52cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:57.620507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb22175-23ec-49e6-8e03-c5df04f52cf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:57.620507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb22175-23ec-49e6-8e03-c5df04f52cf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:04:57.620507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb22175-23ec-49e6-8e03-c5df04f52cf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:57.620507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb22175-23ec-49e6-8e03-c5df04f52cf2 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:57.659738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 844/1662] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:04:57.662661 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb22175-23ec-49e6-8e03-c5df04f52cf2 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:57.673813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8fa88-61dc-41ad-89ee-51c002b6d118 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:57.674610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8fa88-61dc-41ad-89ee-51c002b6d118 service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:57.674968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8fa88-61dc-41ad-89ee-51c002b6d118 service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:57.675317 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8fa88-61dc-41ad-89ee-51c002b6d118 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:57.689237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1fc8fa88-61dc-41ad-89ee-51c002b6d118 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:57.692290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb22175-23ec-49e6-8e03-c5df04f52cf2 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:57.708354 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f361c3-9344-4219-b843-74365970fb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:57.709921 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f361c3-9344-4219-b843-74365970fb79 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:57.712070 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f361c3-9344-4219-b843-74365970fb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:57.712908 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f361c3-9344-4219-b843-74365970fb79 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:57.718094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f361c3-9344-4219-b843-74365970fb79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:57.723741 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f361c3-9344-4219-b843-74365970fb79 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:57.816496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8fa88-61dc-41ad-89ee-51c002b6d118 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:57.819726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8fa88-61dc-41ad-89ee-51c002b6d118 service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:57.833551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 819/1663] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:04:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:04:57.939051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8fa88-61dc-41ad-89ee-51c002b6d118 service nova] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:57.943447 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 845/1664] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:57 2022] POST /identity/v3/projects => generated 365 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:04:57.979681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8fa88-61dc-41ad-89ee-51c002b6d118 service nova] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:58.121620 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf91a4f-3665-4f05-9015-3cbf95a7bec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:58.123691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf91a4f-3665-4f05-9015-3cbf95a7bec5 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:58.124546 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf91a4f-3665-4f05-9015-3cbf95a7bec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:58.125162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf91a4f-3665-4f05-9015-3cbf95a7bec5 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:58.126143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf91a4f-3665-4f05-9015-3cbf95a7bec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:58.129570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf91a4f-3665-4f05-9015-3cbf95a7bec5 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:58.303910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 820/1665] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:04:57 2022] POST /identity/v3/users => generated 283 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:04:58.306309 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f361c3-9344-4219-b843-74365970fb79 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:58.352545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f361c3-9344-4219-b843-74365970fb79 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:58.360880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79afb1-73c8-4066-8330-d257a39644b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:58.365514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79afb1-73c8-4066-8330-d257a39644b0 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:58.368630 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79afb1-73c8-4066-8330-d257a39644b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:58.369456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79afb1-73c8-4066-8330-d257a39644b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:58.373464 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79afb1-73c8-4066-8330-d257a39644b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:58.383814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79afb1-73c8-4066-8330-d257a39644b0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:58.446920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf91a4f-3665-4f05-9015-3cbf95a7bec5 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:58.454296 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 846/1666] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:58.480918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf91a4f-3665-4f05-9015-3cbf95a7bec5 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:58.500449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870f7d0-823f-4951-a819-e6c82152533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:58.501900 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870f7d0-823f-4951-a819-e6c82152533c admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:58.502657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870f7d0-823f-4951-a819-e6c82152533c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:58.503411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870f7d0-823f-4951-a819-e6c82152533c admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:58.505026 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870f7d0-823f-4951-a819-e6c82152533c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:58.508334 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870f7d0-823f-4951-a819-e6c82152533c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:58.548418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79afb1-73c8-4066-8330-d257a39644b0 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:58.560652 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 821/1667] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:04:58.576769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79afb1-73c8-4066-8330-d257a39644b0 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:04:58.585433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044b59d-764d-431b-9998-3a88db32d4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:04:58.586768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044b59d-764d-431b-9998-3a88db32d4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:58.587599 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044b59d-764d-431b-9998-3a88db32d4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:58.588212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044b59d-764d-431b-9998-3a88db32d4c1 admin admin] PATH_INFO: `/v3/projects/fa351520cc344f7cba8c5deb66fc13a1/users/f2e65c45d2684822b212334a6eba77c0/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:58.677588 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044b59d-764d-431b-9998-3a88db32d4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa351520cc344f7cba8c5deb66fc13a1, user_id=f2e65c45d2684822b212334a6eba77c0, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:04:58.705664 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044b59d-764d-431b-9998-3a88db32d4c1 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:04:58.826197 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 847/1668] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:04:58 2022] PUT /identity/v3/projects/fa351520cc344f7cba8c5deb66fc13a1/users/f2e65c45d2684822b212334a6eba77c0/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:04:58.847665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583a016-5138-4a2a-ad79-ca1e8ea9c730 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:04:58.848683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583a016-5138-4a2a-ad79-ca1e8ea9c730 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:04:58.849105 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583a016-5138-4a2a-ad79-ca1e8ea9c730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:04:58.959974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-f583a016-5138-4a2a-ad79-ca1e8ea9c730 None None] MFA Rules not processed for user `f2e65c45d2684822b212334a6eba77c0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:04:58.966187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f583a016-5138-4a2a-ad79-ca1e8ea9c730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:04:59.016356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:04:59.645463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044b59d-764d-431b-9998-3a88db32d4c1 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:04:59.649766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 822/1669] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:04:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 802 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:00.038278 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044b59d-764d-431b-9998-3a88db32d4c1 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:00.054119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161871f9-382c-44b2-8ab3-7cb120c065c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:00.057970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161871f9-382c-44b2-8ab3-7cb120c065c5 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:00.058577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161871f9-382c-44b2-8ab3-7cb120c065c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:00.061145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161871f9-382c-44b2-8ab3-7cb120c065c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:00.062699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161871f9-382c-44b2-8ab3-7cb120c065c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:00.077048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161871f9-382c-44b2-8ab3-7cb120c065c5 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:00.252757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583a016-5138-4a2a-ad79-ca1e8ea9c730 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:00.264752 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 848/1670] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:04:59 2022] POST /identity/v3/projects => generated 369 bytes in 619 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:00.288787 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583a016-5138-4a2a-ad79-ca1e8ea9c730 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:00.461371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09325ee-e9a2-4699-9264-7617c1924a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:00.466226 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09325ee-e9a2-4699-9264-7617c1924a60 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:00.467134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09325ee-e9a2-4699-9264-7617c1924a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:00.469302 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09325ee-e9a2-4699-9264-7617c1924a60 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:00.469968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09325ee-e9a2-4699-9264-7617c1924a60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:00.478078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09325ee-e9a2-4699-9264-7617c1924a60 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:00.671365 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 823/1671] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:00 2022] POST /identity/v3/users => generated 291 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:00.677975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161871f9-382c-44b2-8ab3-7cb120c065c5 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:00.740895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161871f9-382c-44b2-8ab3-7cb120c065c5 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:00.765431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc0cb6-236b-4d81-9bf9-1e8387da6d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:00.766834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc0cb6-236b-4d81-9bf9-1e8387da6d17 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:00.768867 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc0cb6-236b-4d81-9bf9-1e8387da6d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:00.774923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc0cb6-236b-4d81-9bf9-1e8387da6d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:00.778355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc0cb6-236b-4d81-9bf9-1e8387da6d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:00.782201 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc0cb6-236b-4d81-9bf9-1e8387da6d17 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:00.813610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 849/1672] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:00.821622 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09325ee-e9a2-4699-9264-7617c1924a60 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:00.853895 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09325ee-e9a2-4699-9264-7617c1924a60 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:00.862993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453a0c1-7b5f-4312-86f3-4c5d05023674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:00.864678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453a0c1-7b5f-4312-86f3-4c5d05023674 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:00.865522 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453a0c1-7b5f-4312-86f3-4c5d05023674 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:00.866412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453a0c1-7b5f-4312-86f3-4c5d05023674 admin admin] PATH_INFO: `/v3/projects/eef21c8474dc4ad68a1caec248c186d5/users/81b3f9953f3f43379d240765aaa8c0e2/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:00.960018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453a0c1-7b5f-4312-86f3-4c5d05023674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef21c8474dc4ad68a1caec248c186d5, user_id=81b3f9953f3f43379d240765aaa8c0e2, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:00.974417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453a0c1-7b5f-4312-86f3-4c5d05023674 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:00.990223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:01.129223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 824/1673] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:00 2022] PUT /identity/v3/projects/eef21c8474dc4ad68a1caec248c186d5/users/81b3f9953f3f43379d240765aaa8c0e2/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:05:01.136095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc0cb6-236b-4d81-9bf9-1e8387da6d17 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:01.191184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc0cb6-236b-4d81-9bf9-1e8387da6d17 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:01.210129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:01.339224 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d296b-7d5c-4a25-ad4b-2e8576586277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:01.340273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d296b-7d5c-4a25-ad4b-2e8576586277 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:01.345886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d296b-7d5c-4a25-ad4b-2e8576586277 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:01.346773 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d296b-7d5c-4a25-ad4b-2e8576586277 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:01.348407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d296b-7d5c-4a25-ad4b-2e8576586277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:01.357325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d296b-7d5c-4a25-ad4b-2e8576586277 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:01.404031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 850/1674] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:05:01.409960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453a0c1-7b5f-4312-86f3-4c5d05023674 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:01.436661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453a0c1-7b5f-4312-86f3-4c5d05023674 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:01.487588 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442db763-e034-48e1-b8a7-80eb139f743e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:01.493091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442db763-e034-48e1-b8a7-80eb139f743e admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:01.493825 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442db763-e034-48e1-b8a7-80eb139f743e admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:01.504275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442db763-e034-48e1-b8a7-80eb139f743e admin admin] PATH_INFO: `/v3/projects/eef21c8474dc4ad68a1caec248c186d5/users/81b3f9953f3f43379d240765aaa8c0e2/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:01.519032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442db763-e034-48e1-b8a7-80eb139f743e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef21c8474dc4ad68a1caec248c186d5, user_id=81b3f9953f3f43379d240765aaa8c0e2, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:01.525120 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442db763-e034-48e1-b8a7-80eb139f743e admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:01.553468 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 825/1675] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:01 2022] PUT /identity/v3/projects/eef21c8474dc4ad68a1caec248c186d5/users/81b3f9953f3f43379d240765aaa8c0e2/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:01.574007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dd50e-2204-424f-a3d4-6539cdb7c398 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:01.574826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dd50e-2204-424f-a3d4-6539cdb7c398 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:01.575517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dd50e-2204-424f-a3d4-6539cdb7c398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:01.644744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-d39dd50e-2204-424f-a3d4-6539cdb7c398 None None] MFA Rules not processed for user `81b3f9953f3f43379d240765aaa8c0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:01.651820 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d39dd50e-2204-424f-a3d4-6539cdb7c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:01.984833 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 851/1676] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:01 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:01.996482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442db763-e034-48e1-b8a7-80eb139f743e admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:02.000085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-442db763-e034-48e1-b8a7-80eb139f743e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:02.003118 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39dd50e-2204-424f-a3d4-6539cdb7c398 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:02.011422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d39dd50e-2204-424f-a3d4-6539cdb7c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:02.191800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39dd50e-2204-424f-a3d4-6539cdb7c398 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:02.201429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207b557-daf2-4404-a054-de0c834d0a29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:02.202190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207b557-daf2-4404-a054-de0c834d0a29 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:02.202589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207b557-daf2-4404-a054-de0c834d0a29 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:02.202939 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207b557-daf2-4404-a054-de0c834d0a29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:02.206820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442db763-e034-48e1-b8a7-80eb139f743e admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:02.219050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207b557-daf2-4404-a054-de0c834d0a29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:02.222192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207b557-daf2-4404-a054-de0c834d0a29 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:02.224964 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526d56d-7208-43f0-959b-7ab50c9cace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:02.226106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526d56d-7208-43f0-959b-7ab50c9cace5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:02.226634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526d56d-7208-43f0-959b-7ab50c9cace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:02.227204 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526d56d-7208-43f0-959b-7ab50c9cace5 admin admin] PATH_INFO: `/v3/users/d342d63f84d64871a35fd5aad3a254fa` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:02.229133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526d56d-7208-43f0-959b-7ab50c9cace5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d342d63f84d64871a35fd5aad3a254fa)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:02.232900 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526d56d-7208-43f0-959b-7ab50c9cace5 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:02.257919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 852/1677] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:02 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:02.593111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1526d56d-7208-43f0-959b-7ab50c9cace5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd342d63f84d64871a35fd5aad3a254fa'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:02.595931 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1526d56d-7208-43f0-959b-7ab50c9cace5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd342d63f84d64871a35fd5aad3a254fa'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:02.597562 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1526d56d-7208-43f0-959b-7ab50c9cace5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd342d63f84d64871a35fd5aad3a254fa'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:02.623169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1526d56d-7208-43f0-959b-7ab50c9cace5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd342d63f84d64871a35fd5aad3a254fa'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:02.672428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1526d56d-7208-43f0-959b-7ab50c9cace5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd342d63f84d64871a35fd5aad3a254fa'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:02.806347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 826/1678] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:05:01 2022] DELETE /identity/v3/users/d342d63f84d64871a35fd5aad3a254fa => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:02.812349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e1f8b-395a-47a3-9afa-91a1542785d7 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:02.812790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e1f8b-395a-47a3-9afa-91a1542785d7 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:02.813255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e1f8b-395a-47a3-9afa-91a1542785d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:02.871533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-ac0e1f8b-395a-47a3-9afa-91a1542785d7 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:02.873317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ac0e1f8b-395a-47a3-9afa-91a1542785d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:03.000657 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:03.105411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 853/1679] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:05:03.123203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526d56d-7208-43f0-959b-7ab50c9cace5 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:03.125177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1526d56d-7208-43f0-959b-7ab50c9cace5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:03.243707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526d56d-7208-43f0-959b-7ab50c9cace5 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:03.250022 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6647dec-ed14-4530-87a7-8a33d29b71f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:03.251740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6647dec-ed14-4530-87a7-8a33d29b71f7 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:03.252497 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6647dec-ed14-4530-87a7-8a33d29b71f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:03.253078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6647dec-ed14-4530-87a7-8a33d29b71f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:03.272584 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6647dec-ed14-4530-87a7-8a33d29b71f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:03.274256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6647dec-ed14-4530-87a7-8a33d29b71f7 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:03.288637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 827/1680] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:04.449828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066dcb0-d85d-457d-9e37-f15ded472676 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:04.449828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066dcb0-d85d-457d-9e37-f15ded472676 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:04.449828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066dcb0-d85d-457d-9e37-f15ded472676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:04.537551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b066dcb0-d85d-457d-9e37-f15ded472676 None None] MFA Rules not processed for user `c15d08b7eb014af3afc4d0a6e032f740`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:04.538915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b066dcb0-d85d-457d-9e37-f15ded472676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:04.722123 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 854/1681] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:04.726668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61297a-37a5-4d68-9e17-dde6e371a759 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:04.726668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61297a-37a5-4d68-9e17-dde6e371a759 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:04.726668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61297a-37a5-4d68-9e17-dde6e371a759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:04.742973 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:04.794166 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e23a25-c1ad-49b2-ab69-66ad6d50d489 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:04.794648 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e23a25-c1ad-49b2-ab69-66ad6d50d489 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:04.794933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e23a25-c1ad-49b2-ab69-66ad6d50d489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:04.801340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-df61297a-37a5-4d68-9e17-dde6e371a759 None None] MFA Rules not processed for user `c15d08b7eb014af3afc4d0a6e032f740`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:04.812036 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-df61297a-37a5-4d68-9e17-dde6e371a759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:04.857778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-f1e23a25-c1ad-49b2-ab69-66ad6d50d489 None None] MFA Rules not processed for user `a1bc5bf55e9d452b88757ea4080165e0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:04.860086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f1e23a25-c1ad-49b2-ab69-66ad6d50d489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:04.876780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 828/1682] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:05:04.888097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61297a-37a5-4d68-9e17-dde6e371a759 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:04.890004 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-df61297a-37a5-4d68-9e17-dde6e371a759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:05.086869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 855/1683] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:05:05.106901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e23a25-c1ad-49b2-ab69-66ad6d50d489 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:05.108174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f1e23a25-c1ad-49b2-ab69-66ad6d50d489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:05.139321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61297a-37a5-4d68-9e17-dde6e371a759 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:05.164272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a001f4-952b-4e2e-9f56-30d0d290353f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:05.167696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a001f4-952b-4e2e-9f56-30d0d290353f service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:05.168345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a001f4-952b-4e2e-9f56-30d0d290353f service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:05.168632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a001f4-952b-4e2e-9f56-30d0d290353f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:05.177824 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e23a25-c1ad-49b2-ab69-66ad6d50d489 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:05.209936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a001f4-952b-4e2e-9f56-30d0d290353f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:05.211646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5e6ef-6de6-4014-a9af-de00858271b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:05.212399 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5e6ef-6de6-4014-a9af-de00858271b5 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:05.214317 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a001f4-952b-4e2e-9f56-30d0d290353f service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:05.215515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5e6ef-6de6-4014-a9af-de00858271b5 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:05.216202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5e6ef-6de6-4014-a9af-de00858271b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:05.249375 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a5e6ef-6de6-4014-a9af-de00858271b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:05.249375 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a5e6ef-6de6-4014-a9af-de00858271b5 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:05.256314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 829/1684] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:05.297450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 856/1685] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:05.417479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801bea4-9a95-4d1d-b6f0-db704a9f21a5 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:05.418279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801bea4-9a95-4d1d-b6f0-db704a9f21a5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:05.425678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801bea4-9a95-4d1d-b6f0-db704a9f21a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:05.444314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5e6ef-6de6-4014-a9af-de00858271b5 service nova] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:05.449571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-46a5e6ef-6de6-4014-a9af-de00858271b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:05.496269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5e6ef-6de6-4014-a9af-de00858271b5 service nova] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:05.497353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-9801bea4-9a95-4d1d-b6f0-db704a9f21a5 None None] MFA Rules not processed for user `d2738e8ef94a48bb9a0d6b1177598112`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:05.499350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9801bea4-9a95-4d1d-b6f0-db704a9f21a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:05.530688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f69b1-3921-4165-95f5-adc4e3bccaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:05.531436 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f69b1-3921-4165-95f5-adc4e3bccaa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:05.531821 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f69b1-3921-4165-95f5-adc4e3bccaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:05.532161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f69b1-3921-4165-95f5-adc4e3bccaa0 admin admin] PATH_INFO: `/v3/projects/2698a98c09a94ac0a09c50a6961fc431` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:05.534299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f69b1-3921-4165-95f5-adc4e3bccaa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2698a98c09a94ac0a09c50a6961fc431)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:05.536609 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f69b1-3921-4165-95f5-adc4e3bccaa0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:05.721184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 830/1686] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:05.818748 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:05.834567 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-da6f69b1-3921-4165-95f5-adc4e3bccaa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2698a98c09a94ac0a09c50a6961fc431', 'request_id': 'req-da6f69b1-3921-4165-95f5-adc4e3bccaa0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:05.859977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-da6f69b1-3921-4165-95f5-adc4e3bccaa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2698a98c09a94ac0a09c50a6961fc431', 'request_id': 'req-da6f69b1-3921-4165-95f5-adc4e3bccaa0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:05.978343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-da6f69b1-3921-4165-95f5-adc4e3bccaa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2698a98c09a94ac0a09c50a6961fc431', 'request_id': 'req-da6f69b1-3921-4165-95f5-adc4e3bccaa0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:06.020760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-da6f69b1-3921-4165-95f5-adc4e3bccaa0 admin admin] The token cache is being invalidate because project 2698a98c09a94ac0a09c50a6961fc431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:05:06.021374 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-da6f69b1-3921-4165-95f5-adc4e3bccaa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:06.023702 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-da6f69b1-3921-4165-95f5-adc4e3bccaa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:06.030916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 857/1687] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:05:05 2022] DELETE /identity/v3/projects/2698a98c09a94ac0a09c50a6961fc431 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:06.035602 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5fe73-3a26-4840-829a-ea3b9aaad4fc None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:06.036554 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5fe73-3a26-4840-829a-ea3b9aaad4fc None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:06.037059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5fe73-3a26-4840-829a-ea3b9aaad4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:06.092452 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-81b5fe73-3a26-4840-829a-ea3b9aaad4fc None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:06.095182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-81b5fe73-3a26-4840-829a-ea3b9aaad4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:06.306871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 831/1688] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:06.316424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601f7df-8968-4df4-92e5-f8167976e00a None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:06.317057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601f7df-8968-4df4-92e5-f8167976e00a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:06.317497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601f7df-8968-4df4-92e5-f8167976e00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:06.400283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7601f7df-8968-4df4-92e5-f8167976e00a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:06.426844 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7601f7df-8968-4df4-92e5-f8167976e00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:06.470391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 858/1689] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:06.476098 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5fe73-3a26-4840-829a-ea3b9aaad4fc None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:06.543033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5fe73-3a26-4840-829a-ea3b9aaad4fc None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:06.564659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802600f-f612-4be7-85c6-e818116d4d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:06.566984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802600f-f612-4be7-85c6-e818116d4d88 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:06.570254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802600f-f612-4be7-85c6-e818116d4d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:06.570876 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802600f-f612-4be7-85c6-e818116d4d88 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:06.572393 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802600f-f612-4be7-85c6-e818116d4d88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:05:06.573185 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802600f-f612-4be7-85c6-e818116d4d88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:06.596173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802600f-f612-4be7-85c6-e818116d4d88 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:06.645673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 832/1690] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:05:06.652456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601f7df-8968-4df4-92e5-f8167976e00a None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:06.681991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601f7df-8968-4df4-92e5-f8167976e00a None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:06.698673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c180a-2b95-4248-84e7-b220d88b9bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:06.699570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c180a-2b95-4248-84e7-b220d88b9bbf admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:06.699963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c180a-2b95-4248-84e7-b220d88b9bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:06.700280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c180a-2b95-4248-84e7-b220d88b9bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:06.700995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c180a-2b95-4248-84e7-b220d88b9bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:06.707790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c180a-2b95-4248-84e7-b220d88b9bbf admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:06.809260 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcdb37-b511-48c2-98e0-f4295ff35ca9 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:06.809608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcdb37-b511-48c2-98e0-f4295ff35ca9 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:06.809821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcdb37-b511-48c2-98e0-f4295ff35ca9 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:06.811661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 833/1691] 10.222.0.94 () {54 vars in 912 bytes} [Tue Jul 26 15:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:05:06.827977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2a00e-a482-4be0-b3a0-a364f01b1645 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:06.828274 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2a00e-a482-4be0-b3a0-a364f01b1645 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:06.828599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2a00e-a482-4be0-b3a0-a364f01b1645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:06.895807 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 859/1692] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:06 2022] POST /identity/v3/projects => generated 385 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:06.900090 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c180a-2b95-4248-84e7-b220d88b9bbf admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:06.907888 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:06.948194 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-91e2a00e-a482-4be0-b3a0-a364f01b1645 None None] MFA Rules not processed for user `d39fd2708591424ca4892973e7da48ee`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:06.954276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-91e2a00e-a482-4be0-b3a0-a364f01b1645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:07.159509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 834/1693] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 15:05:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:07.163023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c180a-2b95-4248-84e7-b220d88b9bbf admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:07.169153 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2a00e-a482-4be0-b3a0-a364f01b1645 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:07.170229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3ea0f-f144-4ba7-95aa-13bde246c154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:07.170928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3ea0f-f144-4ba7-95aa-13bde246c154 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:07.171212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3ea0f-f144-4ba7-95aa-13bde246c154 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:07.171587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3ea0f-f144-4ba7-95aa-13bde246c154 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:07.172335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3ea0f-f144-4ba7-95aa-13bde246c154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:07.174333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-91e2a00e-a482-4be0-b3a0-a364f01b1645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:07.175109 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3ea0f-f144-4ba7-95aa-13bde246c154 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:07.202152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2a00e-a482-4be0-b3a0-a364f01b1645 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:07.215766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6cade-eff0-4085-a2e1-9fc9ec7f28ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:07.217541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6cade-eff0-4085-a2e1-9fc9ec7f28ae service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:07.218266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6cade-eff0-4085-a2e1-9fc9ec7f28ae service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:07.218960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6cade-eff0-4085-a2e1-9fc9ec7f28ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:07.255435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6cade-eff0-4085-a2e1-9fc9ec7f28ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:07.257244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6cade-eff0-4085-a2e1-9fc9ec7f28ae service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:07.289057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 835/1694] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:05:07.330921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6cade-eff0-4085-a2e1-9fc9ec7f28ae service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:07.331773 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 860/1695] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:06 2022] POST /identity/v3/users => generated 293 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:07.361661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6cade-eff0-4085-a2e1-9fc9ec7f28ae service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:07.388568 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01320a9-72c6-48d4-bc54-d85af3b6f460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:07.389441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01320a9-72c6-48d4-bc54-d85af3b6f460 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:07.389740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01320a9-72c6-48d4-bc54-d85af3b6f460 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:07.390074 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01320a9-72c6-48d4-bc54-d85af3b6f460 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:07.391205 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01320a9-72c6-48d4-bc54-d85af3b6f460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:07.397693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01320a9-72c6-48d4-bc54-d85af3b6f460 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:07.431251 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 836/1696] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:05:07.432989 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3ea0f-f144-4ba7-95aa-13bde246c154 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:07.446507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3ea0f-f144-4ba7-95aa-13bde246c154 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:07.466824 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1183f-5f82-4ed7-83a7-b44ecf5e6d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:07.467539 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1183f-5f82-4ed7-83a7-b44ecf5e6d27 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:07.467825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1183f-5f82-4ed7-83a7-b44ecf5e6d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:07.468101 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1183f-5f82-4ed7-83a7-b44ecf5e6d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:07.468834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1183f-5f82-4ed7-83a7-b44ecf5e6d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:07.474647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1183f-5f82-4ed7-83a7-b44ecf5e6d27 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:07.524797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 861/1697] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:05:07.526972 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01320a9-72c6-48d4-bc54-d85af3b6f460 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:07.563164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01320a9-72c6-48d4-bc54-d85af3b6f460 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:07.575105 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a8095-048a-40a6-810c-28414f883aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:07.576349 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a8095-048a-40a6-810c-28414f883aec admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:07.576813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a8095-048a-40a6-810c-28414f883aec admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:07.578153 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a8095-048a-40a6-810c-28414f883aec admin admin] PATH_INFO: `/v3/projects/301672d4250b44caa2a2442a8a79f72e/users/65e69af7926a40788365dc6b194bc486/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:07.628710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a8095-048a-40a6-810c-28414f883aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=301672d4250b44caa2a2442a8a79f72e, user_id=65e69af7926a40788365dc6b194bc486, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:07.632642 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a8095-048a-40a6-810c-28414f883aec admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:07.814385 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 837/1698] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:07 2022] PUT /identity/v3/projects/301672d4250b44caa2a2442a8a79f72e/users/65e69af7926a40788365dc6b194bc486/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:07.816443 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c403138-2c68-49eb-9dc9-324700fd3f4d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:07.824140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c403138-2c68-49eb-9dc9-324700fd3f4d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:07.824664 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c403138-2c68-49eb-9dc9-324700fd3f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:07.909822 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7c403138-2c68-49eb-9dc9-324700fd3f4d None None] MFA Rules not processed for user `65e69af7926a40788365dc6b194bc486`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:07.909822 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7c403138-2c68-49eb-9dc9-324700fd3f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:08.014279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f058f-866d-44cd-a454-e6acad6aa460 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:08.015248 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f058f-866d-44cd-a454-e6acad6aa460 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:08.016021 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f058f-866d-44cd-a454-e6acad6aa460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:08.266833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-626f058f-866d-44cd-a454-e6acad6aa460 None None] MFA Rules not processed for user `61b3efca1989483d960b9f92d655d39e`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:08.285924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-626f058f-866d-44cd-a454-e6acad6aa460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:08.408379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 862/1699] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 593 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:08.419818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c403138-2c68-49eb-9dc9-324700fd3f4d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:08.447161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:08.661817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 838/1700] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 651 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:08.678065 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0ab24-bef1-49f1-8014-727867d7aec6 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:08.679941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0ab24-bef1-49f1-8014-727867d7aec6 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:08.680479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0ab24-bef1-49f1-8014-727867d7aec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:08.705843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c403138-2c68-49eb-9dc9-324700fd3f4d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:08.735370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89776451-0eec-4754-8bfc-b732dacafdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:08.741181 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89776451-0eec-4754-8bfc-b732dacafdfc admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:08.743319 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89776451-0eec-4754-8bfc-b732dacafdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:08.743868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89776451-0eec-4754-8bfc-b732dacafdfc admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:08.744952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89776451-0eec-4754-8bfc-b732dacafdfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:08.746227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-a3e0ab24-bef1-49f1-8014-727867d7aec6 None None] MFA Rules not processed for user `61b3efca1989483d960b9f92d655d39e`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:08.748491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89776451-0eec-4754-8bfc-b732dacafdfc admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:08.754319 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a3e0ab24-bef1-49f1-8014-727867d7aec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:08.786875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 839/1701] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:05:08.840142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302931-7acb-4e21-9140-0e9fa1dd6d1b None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:08.840629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302931-7acb-4e21-9140-0e9fa1dd6d1b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:08.840988 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a302931-7acb-4e21-9140-0e9fa1dd6d1b None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:08.843384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 840/1702] 10.222.0.94 () {54 vars in 912 bytes} [Tue Jul 26 15:05:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:05:08.858727 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd468b3-30ac-467f-abc1-333804774d5e None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:08.859027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd468b3-30ac-467f-abc1-333804774d5e None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:08.859340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd468b3-30ac-467f-abc1-333804774d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:08.907657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-9dd468b3-30ac-467f-abc1-333804774d5e None None] MFA Rules not processed for user `d39fd2708591424ca4892973e7da48ee`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:08.908971 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9dd468b3-30ac-467f-abc1-333804774d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:08.992031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 863/1703] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:08 2022] POST /identity/v3/projects => generated 385 bytes in 573 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:08.998313 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89776451-0eec-4754-8bfc-b732dacafdfc admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:09.005402 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:09.039793 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 841/1704] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 15:05:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:05:09.061617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd468b3-30ac-467f-abc1-333804774d5e None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:09.065644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9dd468b3-30ac-467f-abc1-333804774d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:09.080387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd468b3-30ac-467f-abc1-333804774d5e None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:09.200829 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9532265-6589-474c-b17e-4b484c589750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:09.201891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9532265-6589-474c-b17e-4b484c589750 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:09.202361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9532265-6589-474c-b17e-4b484c589750 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:09.202785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9532265-6589-474c-b17e-4b484c589750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:09.242812 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9532265-6589-474c-b17e-4b484c589750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:09.245616 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89776451-0eec-4754-8bfc-b732dacafdfc admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:09.247062 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9532265-6589-474c-b17e-4b484c589750 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:09.276740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 842/1705] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:09.281550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4e62c-363f-4136-a1ea-e2f252b0d8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:09.281550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4e62c-363f-4136-a1ea-e2f252b0d8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:09.281550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4e62c-363f-4136-a1ea-e2f252b0d8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:09.281550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4e62c-363f-4136-a1ea-e2f252b0d8d3 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:09.282731 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4e62c-363f-4136-a1ea-e2f252b0d8d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:09.288918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9532265-6589-474c-b17e-4b484c589750 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:09.292010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4e62c-363f-4136-a1ea-e2f252b0d8d3 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:09.292354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b9532265-6589-474c-b17e-4b484c589750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:09.476172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9532265-6589-474c-b17e-4b484c589750 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:09.478987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 864/1706] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:08 2022] POST /identity/v3/users => generated 293 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:09.481157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d704ef2-cdb6-4b82-b800-11179e9b0aee None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:09.483304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d704ef2-cdb6-4b82-b800-11179e9b0aee None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:09.485617 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d704ef2-cdb6-4b82-b800-11179e9b0aee None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:09.495555 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e3ffd-7625-41f7-b858-32334c4d7e73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:09.496293 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e3ffd-7625-41f7-b858-32334c4d7e73 service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:09.496770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e3ffd-7625-41f7-b858-32334c4d7e73 service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:09.497122 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e3ffd-7625-41f7-b858-32334c4d7e73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:09.498194 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 865/1707] 10.222.0.94 () {54 vars in 961 bytes} [Tue Jul 26 15:05:09 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:05:09.499765 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d704ef2-cdb6-4b82-b800-11179e9b0aee None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:09.531438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d704ef2-cdb6-4b82-b800-11179e9b0aee None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:09.549274 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c920cda-4163-41ec-8cd9-af4bc8ff8f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:09.549905 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c920cda-4163-41ec-8cd9-af4bc8ff8f47 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:09.550185 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c920cda-4163-41ec-8cd9-af4bc8ff8f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:09.550457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c920cda-4163-41ec-8cd9-af4bc8ff8f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:09.551380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c920cda-4163-41ec-8cd9-af4bc8ff8f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:09.556022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c920cda-4163-41ec-8cd9-af4bc8ff8f47 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:09.584471 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 866/1708] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:09.586018 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb46ec1-2fa6-4ba9-9b91-350a6b394fab None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:09.586301 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb46ec1-2fa6-4ba9-9b91-350a6b394fab None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:09.586667 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb46ec1-2fa6-4ba9-9b91-350a6b394fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:09.641699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-1fb46ec1-2fa6-4ba9-9b91-350a6b394fab None None] MFA Rules not processed for user `42a0728e2f6249bdb45d6f44fb15aa4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:09.643501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1fb46ec1-2fa6-4ba9-9b91-350a6b394fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:09.655309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e3ffd-7625-41f7-b858-32334c4d7e73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:09.665180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e3ffd-7625-41f7-b858-32334c4d7e73 service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:09.705215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 843/1709] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:09.717870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e3ffd-7625-41f7-b858-32334c4d7e73 service nova] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:09.752129 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e3ffd-7625-41f7-b858-32334c4d7e73 service nova] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:09.768223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357f466-f17f-47a8-b43d-882f7ae096c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:09.770221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357f466-f17f-47a8-b43d-882f7ae096c0 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:09.774388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357f466-f17f-47a8-b43d-882f7ae096c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:09.774388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357f466-f17f-47a8-b43d-882f7ae096c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:09.774388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357f466-f17f-47a8-b43d-882f7ae096c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:09.778557 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357f466-f17f-47a8-b43d-882f7ae096c0 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:09.798612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 844/1710] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:09.808332 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357f466-f17f-47a8-b43d-882f7ae096c0 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:09.833079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 867/1711] 10.222.0.94 () {58 vars in 1058 bytes} [Tue Jul 26 15:05:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:09.843694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb46ec1-2fa6-4ba9-9b91-350a6b394fab None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:09.857657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357f466-f17f-47a8-b43d-882f7ae096c0 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:09.869939 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713b7ca-16d7-408a-b453-216d3fbeb930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:09.871376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb46ec1-2fa6-4ba9-9b91-350a6b394fab None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:09.875038 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713b7ca-16d7-408a-b453-216d3fbeb930 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:09.875425 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713b7ca-16d7-408a-b453-216d3fbeb930 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:09.877702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713b7ca-16d7-408a-b453-216d3fbeb930 admin admin] PATH_INFO: `/v3/projects/ce223560877b42b9b68a14f689ec43fe/users/830b3eef9dec454fa62422e1c8486f2d/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:09.884959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a217b-d617-4c94-a041-0a5b9cc97117 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a0728e2f6249bdb45d6f44fb15aa4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:09.886925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a217b-d617-4c94-a041-0a5b9cc97117 service placement] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:09.887966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a217b-d617-4c94-a041-0a5b9cc97117 service placement] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:09.889510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a217b-d617-4c94-a041-0a5b9cc97117 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:09.896608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5d4a217b-d617-4c94-a041-0a5b9cc97117 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:09.936525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a217b-d617-4c94-a041-0a5b9cc97117 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:09.938330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a217b-d617-4c94-a041-0a5b9cc97117 service placement] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:09.959415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713b7ca-16d7-408a-b453-216d3fbeb930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce223560877b42b9b68a14f689ec43fe, user_id=830b3eef9dec454fa62422e1c8486f2d, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:09.969936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713b7ca-16d7-408a-b453-216d3fbeb930 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:09.974317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 868/1712] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:10.112926 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 845/1713] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:09 2022] PUT /identity/v3/projects/ce223560877b42b9b68a14f689ec43fe/users/830b3eef9dec454fa62422e1c8486f2d/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:10.127306 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e6891-9808-46b1-9ffb-5a6d30d4fbc2 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:10.127904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e6891-9808-46b1-9ffb-5a6d30d4fbc2 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:10.127904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e6891-9808-46b1-9ffb-5a6d30d4fbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:10.220023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-5a3e6891-9808-46b1-9ffb-5a6d30d4fbc2 None None] MFA Rules not processed for user `830b3eef9dec454fa62422e1c8486f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:10.230630 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5a3e6891-9808-46b1-9ffb-5a6d30d4fbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:10.619875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713b7ca-16d7-408a-b453-216d3fbeb930 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:10.627974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 869/1714] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:10.856517 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713b7ca-16d7-408a-b453-216d3fbeb930 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:10.901341 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2a29f-3cce-4ab7-b6b5-ba9841321c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:10.902101 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2a29f-3cce-4ab7-b6b5-ba9841321c34 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:10.902388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2a29f-3cce-4ab7-b6b5-ba9841321c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:10.902658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2a29f-3cce-4ab7-b6b5-ba9841321c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:10.903374 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2a29f-3cce-4ab7-b6b5-ba9841321c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:10.913019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2a29f-3cce-4ab7-b6b5-ba9841321c34 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:11.156257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 846/1715] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:10 2022] POST /identity/v3/projects => generated 383 bytes in 536 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:05:11.159432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e6891-9808-46b1-9ffb-5a6d30d4fbc2 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:11.191727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e6891-9808-46b1-9ffb-5a6d30d4fbc2 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:11.285913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:11.421303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d8663-641c-48cc-b361-9d6115844fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:11.422081 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d8663-641c-48cc-b361-9d6115844fde admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:11.422538 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d8663-641c-48cc-b361-9d6115844fde admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:11.422955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d8663-641c-48cc-b361-9d6115844fde admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:11.423871 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d8663-641c-48cc-b361-9d6115844fde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:11.426098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d8663-641c-48cc-b361-9d6115844fde admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:11.668426 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 870/1716] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:11 2022] POST /identity/v3/users => generated 298 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:05:11.673191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2a29f-3cce-4ab7-b6b5-ba9841321c34 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:11.690265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2a29f-3cce-4ab7-b6b5-ba9841321c34 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:11.701389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:11.714744 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af566a7-20d3-47ad-8052-16c35429af8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:11.715410 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af566a7-20d3-47ad-8052-16c35429af8f admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:11.715743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af566a7-20d3-47ad-8052-16c35429af8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:11.716003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af566a7-20d3-47ad-8052-16c35429af8f admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:11.716743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af566a7-20d3-47ad-8052-16c35429af8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:11.718449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af566a7-20d3-47ad-8052-16c35429af8f admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:11.753288 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 847/1717] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:05:11.758043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d8663-641c-48cc-b361-9d6115844fde admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:11.772642 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d8663-641c-48cc-b361-9d6115844fde admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:11.789133 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c07591-f2ed-4b6f-8dc9-084bbc30d2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:11.790028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c07591-f2ed-4b6f-8dc9-084bbc30d2a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:11.790475 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c07591-f2ed-4b6f-8dc9-084bbc30d2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:11.791499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c07591-f2ed-4b6f-8dc9-084bbc30d2a3 admin admin] PATH_INFO: `/v3/projects/6b715f9142654bcabcda78a614862896/users/870b500ece3d425185a8c8d28524a27a/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:11.840643 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c07591-f2ed-4b6f-8dc9-084bbc30d2a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b715f9142654bcabcda78a614862896, user_id=870b500ece3d425185a8c8d28524a27a, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:11.843583 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c07591-f2ed-4b6f-8dc9-084bbc30d2a3 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:12.065251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 871/1718] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:11 2022] PUT /identity/v3/projects/6b715f9142654bcabcda78a614862896/users/870b500ece3d425185a8c8d28524a27a/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:12.068151 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af566a7-20d3-47ad-8052-16c35429af8f admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:12.126550 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af566a7-20d3-47ad-8052-16c35429af8f admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:12.284460 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c07591-f2ed-4b6f-8dc9-084bbc30d2a3 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:12.287961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-20c07591-f2ed-4b6f-8dc9-084bbc30d2a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:12.408969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda71773-3a13-4203-aa42-81b9fdbd2bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:12.409629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda71773-3a13-4203-aa42-81b9fdbd2bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:12.410475 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda71773-3a13-4203-aa42-81b9fdbd2bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:12.410763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda71773-3a13-4203-aa42-81b9fdbd2bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:12.416963 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda71773-3a13-4203-aa42-81b9fdbd2bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:12.419176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda71773-3a13-4203-aa42-81b9fdbd2bd9 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:12.476716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 848/1719] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:12.489618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda71773-3a13-4203-aa42-81b9fdbd2bd9 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:12.527533 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda71773-3a13-4203-aa42-81b9fdbd2bd9 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:12.542976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af222a91-6790-4b84-91f3-a4bad7ce82b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:12.543882 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af222a91-6790-4b84-91f3-a4bad7ce82b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:12.544416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af222a91-6790-4b84-91f3-a4bad7ce82b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:12.544840 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af222a91-6790-4b84-91f3-a4bad7ce82b7 admin admin] PATH_INFO: `/v3/projects/6b715f9142654bcabcda78a614862896/users/870b500ece3d425185a8c8d28524a27a/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:12.550372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af222a91-6790-4b84-91f3-a4bad7ce82b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b715f9142654bcabcda78a614862896, user_id=870b500ece3d425185a8c8d28524a27a, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:12.554967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af222a91-6790-4b84-91f3-a4bad7ce82b7 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:12.567678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c07591-f2ed-4b6f-8dc9-084bbc30d2a3 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:12.581932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 849/1720] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:12 2022] PUT /identity/v3/projects/6b715f9142654bcabcda78a614862896/users/870b500ece3d425185a8c8d28524a27a/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:12.585076 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfbc76-be7e-4d90-8aa2-daf2760c31f5 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:12.585368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfbc76-be7e-4d90-8aa2-daf2760c31f5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:12.585655 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfbc76-be7e-4d90-8aa2-daf2760c31f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:12.649432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-50cfbc76-be7e-4d90-8aa2-daf2760c31f5 None None] MFA Rules not processed for user `870b500ece3d425185a8c8d28524a27a`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:12.651850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-50cfbc76-be7e-4d90-8aa2-daf2760c31f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:12.663138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c3e0a-d71b-4c4a-9f7f-8fe83846d22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:12.663937 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c3e0a-d71b-4c4a-9f7f-8fe83846d22e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:12.674786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c3e0a-d71b-4c4a-9f7f-8fe83846d22e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:12.675198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c3e0a-d71b-4c4a-9f7f-8fe83846d22e admin admin] PATH_INFO: `/v3/users/f2e65c45d2684822b212334a6eba77c0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:12.677418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c3e0a-d71b-4c4a-9f7f-8fe83846d22e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2e65c45d2684822b212334a6eba77c0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:12.680755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c3e0a-d71b-4c4a-9f7f-8fe83846d22e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:12.884040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 850/1721] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:05:12.909110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfbc76-be7e-4d90-8aa2-daf2760c31f5 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:12.912558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-50cfbc76-be7e-4d90-8aa2-daf2760c31f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:13.064466 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6c3c3e0a-d71b-4c4a-9f7f-8fe83846d22e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2e65c45d2684822b212334a6eba77c0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:13.109773 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfbc76-be7e-4d90-8aa2-daf2760c31f5 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:13.113696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6c3c3e0a-d71b-4c4a-9f7f-8fe83846d22e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2e65c45d2684822b212334a6eba77c0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:13.121999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c9cca-8df8-4f0f-a3d1-400e5bd39c23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:13.123962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c9cca-8df8-4f0f-a3d1-400e5bd39c23 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:13.125350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c9cca-8df8-4f0f-a3d1-400e5bd39c23 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:13.127195 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c9cca-8df8-4f0f-a3d1-400e5bd39c23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:13.161050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c9cca-8df8-4f0f-a3d1-400e5bd39c23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:13.168898 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c9cca-8df8-4f0f-a3d1-400e5bd39c23 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:13.215039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 851/1722] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:05:13.258682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6c3c3e0a-d71b-4c4a-9f7f-8fe83846d22e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2e65c45d2684822b212334a6eba77c0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:13.311016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6c3c3e0a-d71b-4c4a-9f7f-8fe83846d22e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2e65c45d2684822b212334a6eba77c0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:13.323703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6c3c3e0a-d71b-4c4a-9f7f-8fe83846d22e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2e65c45d2684822b212334a6eba77c0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:13.346878 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 872/1723] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:05:12 2022] DELETE /identity/v3/users/f2e65c45d2684822b212334a6eba77c0 => generated 0 bytes in 1062 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:13.350049 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd07154-225d-4208-90f6-622234a2e3d2 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:13.350397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd07154-225d-4208-90f6-622234a2e3d2 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:13.350678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd07154-225d-4208-90f6-622234a2e3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:13.404463 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-0cd07154-225d-4208-90f6-622234a2e3d2 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:13.409826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0cd07154-225d-4208-90f6-622234a2e3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:13.614083 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cec3e-f7ce-4250-85c3-7c940204925c None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:13.614847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cec3e-f7ce-4250-85c3-7c940204925c None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:13.615342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cec3e-f7ce-4250-85c3-7c940204925c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:13.668670 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 852/1724] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:13.683025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd07154-225d-4208-90f6-622234a2e3d2 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:13.683938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7e5cec3e-f7ce-4250-85c3-7c940204925c None None] MFA Rules not processed for user `61b3efca1989483d960b9f92d655d39e`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:13.685110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0cd07154-225d-4208-90f6-622234a2e3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:13.688332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7e5cec3e-f7ce-4250-85c3-7c940204925c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:13.792357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:13.907356 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 873/1725] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:13.956297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd07154-225d-4208-90f6-622234a2e3d2 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:13.968301 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325559d4-1504-4e0b-b69c-244691690edd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:13.969774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325559d4-1504-4e0b-b69c-244691690edd service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:13.969774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325559d4-1504-4e0b-b69c-244691690edd service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:13.969774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325559d4-1504-4e0b-b69c-244691690edd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:13.981521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:13.988321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325559d4-1504-4e0b-b69c-244691690edd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:13.996908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325559d4-1504-4e0b-b69c-244691690edd service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:14.020445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 853/1726] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:14.595534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635abf8-3094-4921-a074-12769eeccc51 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:14.596031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635abf8-3094-4921-a074-12769eeccc51 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:14.596105 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635abf8-3094-4921-a074-12769eeccc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:14.638127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-e635abf8-3094-4921-a074-12769eeccc51 None None] MFA Rules not processed for user `a1bc5bf55e9d452b88757ea4080165e0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:14.649885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e635abf8-3094-4921-a074-12769eeccc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:14.946259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 874/1727] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:15.804610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325559d4-1504-4e0b-b69c-244691690edd service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:15.807163 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-325559d4-1504-4e0b-b69c-244691690edd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:15.837096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325559d4-1504-4e0b-b69c-244691690edd service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:15.852607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:15.854048 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:15.857381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:15.857876 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] PATH_INFO: `/v3/projects/fa351520cc344f7cba8c5deb66fc13a1` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:15.869625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa351520cc344f7cba8c5deb66fc13a1)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:15.869625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:16.224177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa351520cc344f7cba8c5deb66fc13a1', 'request_id': 'req-88d25c95-064a-4499-9a58-47f2f6b5055e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:16.243552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa351520cc344f7cba8c5deb66fc13a1', 'request_id': 'req-88d25c95-064a-4499-9a58-47f2f6b5055e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:16.271078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:16.289026 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa351520cc344f7cba8c5deb66fc13a1', 'request_id': 'req-88d25c95-064a-4499-9a58-47f2f6b5055e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:16.452687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:16.460482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] The token cache is being invalidate because project fa351520cc344f7cba8c5deb66fc13a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:05:16.465310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:16.465310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:16.466887 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 854/1728] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:05:15 2022] DELETE /identity/v3/projects/fa351520cc344f7cba8c5deb66fc13a1 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:16.472514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635abf8-3094-4921-a074-12769eeccc51 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:16.500175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635abf8-3094-4921-a074-12769eeccc51 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:16.725149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7775d-9f82-4a1f-a872-cbde0af1657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:16.726074 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7775d-9f82-4a1f-a872-cbde0af1657d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:16.726363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7775d-9f82-4a1f-a872-cbde0af1657d admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:16.726645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7775d-9f82-4a1f-a872-cbde0af1657d admin admin] PATH_INFO: `/v3/users/81b3f9953f3f43379d240765aaa8c0e2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:16.729526 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7775d-9f82-4a1f-a872-cbde0af1657d admin admin] RBAC: Authorizing `identity:delete_user(user_id=81b3f9953f3f43379d240765aaa8c0e2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:16.732405 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7775d-9f82-4a1f-a872-cbde0af1657d admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:17.112509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8fb7775d-9f82-4a1f-a872-cbde0af1657d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81b3f9953f3f43379d240765aaa8c0e2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:17.148355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8fb7775d-9f82-4a1f-a872-cbde0af1657d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81b3f9953f3f43379d240765aaa8c0e2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:17.176316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:17.179775 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:17.270249 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8fb7775d-9f82-4a1f-a872-cbde0af1657d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81b3f9953f3f43379d240765aaa8c0e2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:17.347445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8fb7775d-9f82-4a1f-a872-cbde0af1657d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81b3f9953f3f43379d240765aaa8c0e2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:17.351527 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8fb7775d-9f82-4a1f-a872-cbde0af1657d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81b3f9953f3f43379d240765aaa8c0e2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:17.424890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 875/1729] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:05:16 2022] DELETE /identity/v3/users/81b3f9953f3f43379d240765aaa8c0e2 => generated 0 bytes in 951 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:05:17.566559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d25c95-064a-4499-9a58-47f2f6b5055e admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:17.892218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122de7e-bda6-4ae8-9279-5f3dedfa7349 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:17.895257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122de7e-bda6-4ae8-9279-5f3dedfa7349 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:17.897207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122de7e-bda6-4ae8-9279-5f3dedfa7349 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:17.897733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122de7e-bda6-4ae8-9279-5f3dedfa7349 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:17.905416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8122de7e-bda6-4ae8-9279-5f3dedfa7349 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:18.139669 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca0035-dac6-4705-9263-3c87b110c176 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:18.140726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca0035-dac6-4705-9263-3c87b110c176 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:18.141195 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca0035-dac6-4705-9263-3c87b110c176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:18.308833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122de7e-bda6-4ae8-9279-5f3dedfa7349 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:18.308833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122de7e-bda6-4ae8-9279-5f3dedfa7349 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:18.308833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 855/1730] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 1055 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:18.310836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-49ca0035-dac6-4705-9263-3c87b110c176 None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:18.310836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-49ca0035-dac6-4705-9263-3c87b110c176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:18.427962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 876/1731] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 15:05:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:05:19.017849 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122de7e-bda6-4ae8-9279-5f3dedfa7349 service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:19.020133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8122de7e-bda6-4ae8-9279-5f3dedfa7349 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:19.097527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:19.392515 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122de7e-bda6-4ae8-9279-5f3dedfa7349 service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:19.403258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0733339-64fd-4f31-a076-99dc9136603f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:19.404424 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0733339-64fd-4f31-a076-99dc9136603f service swift] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:19.404926 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0733339-64fd-4f31-a076-99dc9136603f service swift] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:19.405347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0733339-64fd-4f31-a076-99dc9136603f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:19.430069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0733339-64fd-4f31-a076-99dc9136603f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:19.432654 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0733339-64fd-4f31-a076-99dc9136603f service swift] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:19.498742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 856/1732] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:05:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:05:20.324756 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca0035-dac6-4705-9263-3c87b110c176 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:20.329780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-49ca0035-dac6-4705-9263-3c87b110c176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:20.401001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:20.637483 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca0035-dac6-4705-9263-3c87b110c176 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:20.653251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96483aa8-1602-4152-b6da-da45fe2fe570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:20.654204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96483aa8-1602-4152-b6da-da45fe2fe570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:20.657413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96483aa8-1602-4152-b6da-da45fe2fe570 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:20.657413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96483aa8-1602-4152-b6da-da45fe2fe570 admin admin] PATH_INFO: `/v3/projects/eef21c8474dc4ad68a1caec248c186d5` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:20.658537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96483aa8-1602-4152-b6da-da45fe2fe570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eef21c8474dc4ad68a1caec248c186d5)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:20.664424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96483aa8-1602-4152-b6da-da45fe2fe570 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:20.958212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-96483aa8-1602-4152-b6da-da45fe2fe570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eef21c8474dc4ad68a1caec248c186d5', 'request_id': 'req-96483aa8-1602-4152-b6da-da45fe2fe570'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:20.984153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-96483aa8-1602-4152-b6da-da45fe2fe570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eef21c8474dc4ad68a1caec248c186d5', 'request_id': 'req-96483aa8-1602-4152-b6da-da45fe2fe570'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:21.120148 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-96483aa8-1602-4152-b6da-da45fe2fe570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eef21c8474dc4ad68a1caec248c186d5', 'request_id': 'req-96483aa8-1602-4152-b6da-da45fe2fe570'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:21.139122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-96483aa8-1602-4152-b6da-da45fe2fe570 admin admin] The token cache is being invalidate because project eef21c8474dc4ad68a1caec248c186d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:05:21.139479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-96483aa8-1602-4152-b6da-da45fe2fe570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:21.140726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-96483aa8-1602-4152-b6da-da45fe2fe570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:21.146543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 877/1733] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:05:20 2022] DELETE /identity/v3/projects/eef21c8474dc4ad68a1caec248c186d5 => generated 0 bytes in 823 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:21.153131 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70965c00-503e-4436-a5cf-3cf7b3e49d04 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:21.153652 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70965c00-503e-4436-a5cf-3cf7b3e49d04 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:21.153743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70965c00-503e-4436-a5cf-3cf7b3e49d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:21.221578 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-70965c00-503e-4436-a5cf-3cf7b3e49d04 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:21.227109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-70965c00-503e-4436-a5cf-3cf7b3e49d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:21.521937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 857/1734] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:21.530234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617175a-f522-4fef-ba84-fd50d9e427e6 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:21.530912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617175a-f522-4fef-ba84-fd50d9e427e6 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:21.531041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617175a-f522-4fef-ba84-fd50d9e427e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:21.601750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c617175a-f522-4fef-ba84-fd50d9e427e6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:21.615870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c617175a-f522-4fef-ba84-fd50d9e427e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:21.647497 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 878/1735] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:21.653523 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70965c00-503e-4436-a5cf-3cf7b3e49d04 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:21.673609 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70965c00-503e-4436-a5cf-3cf7b3e49d04 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:21.689043 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb123f5-15db-43a7-b540-9b0521f8b361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:21.696183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb123f5-15db-43a7-b540-9b0521f8b361 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:21.696183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb123f5-15db-43a7-b540-9b0521f8b361 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:21.696183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb123f5-15db-43a7-b540-9b0521f8b361 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:21.696183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb123f5-15db-43a7-b540-9b0521f8b361 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:05:21.696183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb123f5-15db-43a7-b540-9b0521f8b361 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:21.699922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb123f5-15db-43a7-b540-9b0521f8b361 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:21.765272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 858/1736] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:05:21.765366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617175a-f522-4fef-ba84-fd50d9e427e6 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:21.779008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617175a-f522-4fef-ba84-fd50d9e427e6 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:21.789954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaa125-be16-482f-ad54-9bd2d6412742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:21.791661 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaa125-be16-482f-ad54-9bd2d6412742 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:21.797391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaa125-be16-482f-ad54-9bd2d6412742 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:21.797391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aaa125-be16-482f-ad54-9bd2d6412742 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:21.823296 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aaa125-be16-482f-ad54-9bd2d6412742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:21.834835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aaa125-be16-482f-ad54-9bd2d6412742 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:21.916750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117e1c2-70ea-4d5e-b8f9-d99c77622432 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:21.917065 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117e1c2-70ea-4d5e-b8f9-d99c77622432 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:21.917308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117e1c2-70ea-4d5e-b8f9-d99c77622432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:21.970401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-f117e1c2-70ea-4d5e-b8f9-d99c77622432 None None] MFA Rules not processed for user `d39fd2708591424ca4892973e7da48ee`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:21.974865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f117e1c2-70ea-4d5e-b8f9-d99c77622432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:22.032904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 879/1737] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:21 2022] POST /identity/v3/projects => generated 377 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:22.037375 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaa125-be16-482f-ad54-9bd2d6412742 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:22.253674 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 859/1738] 10.222.0.102 () {58 vars in 1016 bytes} [Tue Jul 26 15:05:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:22.264190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aaa125-be16-482f-ad54-9bd2d6412742 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:22.285611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117e1c2-70ea-4d5e-b8f9-d99c77622432 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:22.287658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f117e1c2-70ea-4d5e-b8f9-d99c77622432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:22.293197 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f41a4-9c43-4a54-8da6-e286b0a5f476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:22.295340 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f41a4-9c43-4a54-8da6-e286b0a5f476 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:22.295340 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f41a4-9c43-4a54-8da6-e286b0a5f476 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:22.295736 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f41a4-9c43-4a54-8da6-e286b0a5f476 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:22.297844 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f41a4-9c43-4a54-8da6-e286b0a5f476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:22.304761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f41a4-9c43-4a54-8da6-e286b0a5f476 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:22.320198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117e1c2-70ea-4d5e-b8f9-d99c77622432 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:22.483884 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949c119-2378-484c-9993-b0e21c01cadd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:22.486108 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949c119-2378-484c-9993-b0e21c01cadd service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:22.486885 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949c119-2378-484c-9993-b0e21c01cadd service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:22.487695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949c119-2378-484c-9993-b0e21c01cadd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:22.532852 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 880/1739] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:22 2022] POST /identity/v3/users => generated 289 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:22.537156 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f41a4-9c43-4a54-8da6-e286b0a5f476 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:22.551088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949c119-2378-484c-9993-b0e21c01cadd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:22.556526 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949c119-2378-484c-9993-b0e21c01cadd service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:22.589967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 860/1740] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:05:22.589967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949c119-2378-484c-9993-b0e21c01cadd service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:22.594336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f41a4-9c43-4a54-8da6-e286b0a5f476 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:22.610776 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff503b0-c060-4f17-9857-2691ece792f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:22.612022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff503b0-c060-4f17-9857-2691ece792f3 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:22.612500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff503b0-c060-4f17-9857-2691ece792f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:22.614170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff503b0-c060-4f17-9857-2691ece792f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:22.618023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949c119-2378-484c-9993-b0e21c01cadd service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:22.644681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa6b9b-b76e-4a6e-8282-6f991807ad8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:22.645688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa6b9b-b76e-4a6e-8282-6f991807ad8e admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:22.647962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa6b9b-b76e-4a6e-8282-6f991807ad8e admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:22.648508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa6b9b-b76e-4a6e-8282-6f991807ad8e admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:22.650996 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa6b9b-b76e-4a6e-8282-6f991807ad8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:22.658583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa6b9b-b76e-4a6e-8282-6f991807ad8e admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:22.690843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff503b0-c060-4f17-9857-2691ece792f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:22.694506 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 861/1741] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:22.703316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa6b9b-b76e-4a6e-8282-6f991807ad8e admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:22.704284 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff503b0-c060-4f17-9857-2691ece792f3 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:22.738686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 881/1742] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:22.751821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa6b9b-b76e-4a6e-8282-6f991807ad8e admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:22.763766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293fb3a-1a97-48ea-8766-d5a0a579c402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:22.766020 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293fb3a-1a97-48ea-8766-d5a0a579c402 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:22.766985 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293fb3a-1a97-48ea-8766-d5a0a579c402 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:22.767923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293fb3a-1a97-48ea-8766-d5a0a579c402 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:22.769682 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293fb3a-1a97-48ea-8766-d5a0a579c402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:22.774594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293fb3a-1a97-48ea-8766-d5a0a579c402 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:22.835893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 862/1743] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:22.841345 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff503b0-c060-4f17-9857-2691ece792f3 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:22.865991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff503b0-c060-4f17-9857-2691ece792f3 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:22.880370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8386eed-13cd-43af-91b6-6f17e7212b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:22.881739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8386eed-13cd-43af-91b6-6f17e7212b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:22.882272 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8386eed-13cd-43af-91b6-6f17e7212b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:22.882911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8386eed-13cd-43af-91b6-6f17e7212b8a admin admin] PATH_INFO: `/v3/projects/37a36b3e3284428a849f6b4e8cf26b11/users/6537f456e0254b558c4d304e55f39d70/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:22.916352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8386eed-13cd-43af-91b6-6f17e7212b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a36b3e3284428a849f6b4e8cf26b11, user_id=6537f456e0254b558c4d304e55f39d70, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:22.920637 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8386eed-13cd-43af-91b6-6f17e7212b8a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:23.014145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 882/1744] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:22 2022] PUT /identity/v3/projects/37a36b3e3284428a849f6b4e8cf26b11/users/6537f456e0254b558c4d304e55f39d70/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:23.015627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395680d-2b11-488b-a07b-bd778d4e734e None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:23.015922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395680d-2b11-488b-a07b-bd778d4e734e None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:23.016203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395680d-2b11-488b-a07b-bd778d4e734e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:23.134002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8386eed-13cd-43af-91b6-6f17e7212b8a admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:23.134002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c8386eed-13cd-43af-91b6-6f17e7212b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:23.175994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-7395680d-2b11-488b-a07b-bd778d4e734e None None] MFA Rules not processed for user `6537f456e0254b558c4d304e55f39d70`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:23.178770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:23.192833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-7395680d-2b11-488b-a07b-bd778d4e734e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:23.241661 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8386eed-13cd-43af-91b6-6f17e7212b8a admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:23.281144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8e7f6-4b92-46cb-b7d8-786cfc95ae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:23.283132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8e7f6-4b92-46cb-b7d8-786cfc95ae34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:23.283513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8e7f6-4b92-46cb-b7d8-786cfc95ae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:23.283813 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8e7f6-4b92-46cb-b7d8-786cfc95ae34 admin admin] PATH_INFO: `/v3/users/3c5d9ba77968401da3c87d08b6b6027b` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:23.288310 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8e7f6-4b92-46cb-b7d8-786cfc95ae34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c5d9ba77968401da3c87d08b6b6027b)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:23.291203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8e7f6-4b92-46cb-b7d8-786cfc95ae34 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:23.439916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 863/1745] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:23.445073 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395680d-2b11-488b-a07b-bd778d4e734e None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:23.458424 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395680d-2b11-488b-a07b-bd778d4e734e None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:23.497324 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed2c19-4559-4b89-b4e6-afa3dc469100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:23.499131 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed2c19-4559-4b89-b4e6-afa3dc469100 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:23.499810 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed2c19-4559-4b89-b4e6-afa3dc469100 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:23.500169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed2c19-4559-4b89-b4e6-afa3dc469100 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:23.500902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed2c19-4559-4b89-b4e6-afa3dc469100 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:23.507363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed2c19-4559-4b89-b4e6-afa3dc469100 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:23.692537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-44b8e7f6-4b92-46cb-b7d8-786cfc95ae34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c5d9ba77968401da3c87d08b6b6027b'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:23.729255 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 864/1746] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:23 2022] POST /identity/v3/projects => generated 377 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:05:23.733095 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed2c19-4559-4b89-b4e6-afa3dc469100 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:23.750307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-44b8e7f6-4b92-46cb-b7d8-786cfc95ae34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c5d9ba77968401da3c87d08b6b6027b'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:23.861189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed2c19-4559-4b89-b4e6-afa3dc469100 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:23.872887 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-44b8e7f6-4b92-46cb-b7d8-786cfc95ae34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c5d9ba77968401da3c87d08b6b6027b'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:23.879034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f9246-c59c-4646-9153-132534cfe260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:23.883848 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f9246-c59c-4646-9153-132534cfe260 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:23.883848 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f9246-c59c-4646-9153-132534cfe260 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:23.883848 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f9246-c59c-4646-9153-132534cfe260 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:23.886531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f9246-c59c-4646-9153-132534cfe260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:23.891267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f9246-c59c-4646-9153-132534cfe260 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:23.947506 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-44b8e7f6-4b92-46cb-b7d8-786cfc95ae34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c5d9ba77968401da3c87d08b6b6027b'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:23.947506 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-44b8e7f6-4b92-46cb-b7d8-786cfc95ae34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c5d9ba77968401da3c87d08b6b6027b'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:23.981020 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 883/1747] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:05:23 2022] DELETE /identity/v3/users/3c5d9ba77968401da3c87d08b6b6027b => generated 0 bytes in 856 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:23.988084 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33177e2-c501-40e6-98ce-ff98d29d94e0 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:23.988378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33177e2-c501-40e6-98ce-ff98d29d94e0 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:23.988662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33177e2-c501-40e6-98ce-ff98d29d94e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:23.999710 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:24.044689 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 865/1748] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:23 2022] POST /identity/v3/users => generated 295 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:24.045730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f9246-c59c-4646-9153-132534cfe260 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:24.053444 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f76f9246-c59c-4646-9153-132534cfe260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:24.066197 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-e33177e2-c501-40e6-98ce-ff98d29d94e0 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:24.075499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e33177e2-c501-40e6-98ce-ff98d29d94e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:24.114151 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f9246-c59c-4646-9153-132534cfe260 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:24.135118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:24.297966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cd6a8-ed33-4707-82d0-6c363bc6ee71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:24.297966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cd6a8-ed33-4707-82d0-6c363bc6ee71 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:24.298338 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cd6a8-ed33-4707-82d0-6c363bc6ee71 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:24.298546 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cd6a8-ed33-4707-82d0-6c363bc6ee71 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:24.314995 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674cd6a8-ed33-4707-82d0-6c363bc6ee71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:24.322464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674cd6a8-ed33-4707-82d0-6c363bc6ee71 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:24.397695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 884/1750] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:24.401716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 866/1749] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:24.405799 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33177e2-c501-40e6-98ce-ff98d29d94e0 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:24.432847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33177e2-c501-40e6-98ce-ff98d29d94e0 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:24.443265 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e856e-d32e-4827-aaba-071c3556e066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:24.445998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e856e-d32e-4827-aaba-071c3556e066 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:24.446438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e856e-d32e-4827-aaba-071c3556e066 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:24.447377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e856e-d32e-4827-aaba-071c3556e066 admin admin] PATH_INFO: `/v3/projects/65ab32d45feb4576afc7be0fb6fece83/users/c989d519e87d4e4bb3ccd8c5b6fca8a0/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:24.466366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cd6a8-ed33-4707-82d0-6c363bc6ee71 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:24.472525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-674cd6a8-ed33-4707-82d0-6c363bc6ee71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:24.479668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e856e-d32e-4827-aaba-071c3556e066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65ab32d45feb4576afc7be0fb6fece83, user_id=c989d519e87d4e4bb3ccd8c5b6fca8a0, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:24.493781 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e856e-d32e-4827-aaba-071c3556e066 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:24.625386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 885/1751] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:24 2022] PUT /identity/v3/projects/65ab32d45feb4576afc7be0fb6fece83/users/c989d519e87d4e4bb3ccd8c5b6fca8a0/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:24.630599 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e856e-d32e-4827-aaba-071c3556e066 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:24.760952 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cd6a8-ed33-4707-82d0-6c363bc6ee71 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:24.962162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc83f2-7d6c-4df9-bdca-f83e6fbf5a5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:24.962890 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc83f2-7d6c-4df9-bdca-f83e6fbf5a5c service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:24.963182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc83f2-7d6c-4df9-bdca-f83e6fbf5a5c service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:24.963506 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc83f2-7d6c-4df9-bdca-f83e6fbf5a5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:24.979394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc83f2-7d6c-4df9-bdca-f83e6fbf5a5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:24.981043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e856e-d32e-4827-aaba-071c3556e066 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:24.981622 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc83f2-7d6c-4df9-bdca-f83e6fbf5a5c service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:25.007399 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bff71-6bc3-4efc-a154-880344525455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:25.016107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bff71-6bc3-4efc-a154-880344525455 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:25.016107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bff71-6bc3-4efc-a154-880344525455 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:25.016107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bff71-6bc3-4efc-a154-880344525455 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:25.016107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bff71-6bc3-4efc-a154-880344525455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:25.017302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bff71-6bc3-4efc-a154-880344525455 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:25.046794 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 867/1752] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:05:25.066629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 886/1753] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:05:25.070682 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc83f2-7d6c-4df9-bdca-f83e6fbf5a5c service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:25.101013 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc83f2-7d6c-4df9-bdca-f83e6fbf5a5c service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:25.113327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83b6cd-8e4e-45f5-9a93-1cbb486fec29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:25.114430 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83b6cd-8e4e-45f5-9a93-1cbb486fec29 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:25.114817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83b6cd-8e4e-45f5-9a93-1cbb486fec29 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:25.115275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83b6cd-8e4e-45f5-9a93-1cbb486fec29 admin admin] PATH_INFO: `/v3/projects/65ab32d45feb4576afc7be0fb6fece83/users/c989d519e87d4e4bb3ccd8c5b6fca8a0/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:25.123055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83b6cd-8e4e-45f5-9a93-1cbb486fec29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65ab32d45feb4576afc7be0fb6fece83, user_id=c989d519e87d4e4bb3ccd8c5b6fca8a0, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:25.126482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83b6cd-8e4e-45f5-9a93-1cbb486fec29 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:25.143157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 868/1754] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:25 2022] PUT /identity/v3/projects/65ab32d45feb4576afc7be0fb6fece83/users/c989d519e87d4e4bb3ccd8c5b6fca8a0/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:25.173550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12212b-0f28-4a3b-8b90-bf9ad645ceda None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:25.174132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12212b-0f28-4a3b-8b90-bf9ad645ceda None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:25.174701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12212b-0f28-4a3b-8b90-bf9ad645ceda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:25.281973 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-cf12212b-0f28-4a3b-8b90-bf9ad645ceda None None] MFA Rules not processed for user `c989d519e87d4e4bb3ccd8c5b6fca8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:25.298961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-cf12212b-0f28-4a3b-8b90-bf9ad645ceda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:25.529558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 887/1755] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:25.542578 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83b6cd-8e4e-45f5-9a93-1cbb486fec29 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:25.550858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-de83b6cd-8e4e-45f5-9a93-1cbb486fec29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:25.697197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:25.714986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83b6cd-8e4e-45f5-9a93-1cbb486fec29 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:25.731329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed4278-5b6b-413d-84ab-4d2bfe2a2c00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:25.732790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed4278-5b6b-413d-84ab-4d2bfe2a2c00 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:25.732790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed4278-5b6b-413d-84ab-4d2bfe2a2c00 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:25.735284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed4278-5b6b-413d-84ab-4d2bfe2a2c00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:25.743279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-16ed4278-5b6b-413d-84ab-4d2bfe2a2c00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:25.914357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed4278-5b6b-413d-84ab-4d2bfe2a2c00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:25.929299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed4278-5b6b-413d-84ab-4d2bfe2a2c00 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:25.941095 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 869/1756] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:05:25.966193 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:26.386515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f129b-7b86-412e-abd5-15d3a45495d3 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:26.387229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f129b-7b86-412e-abd5-15d3a45495d3 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:26.387229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f129b-7b86-412e-abd5-15d3a45495d3 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:26.388647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 888/1757] 10.222.0.94 () {54 vars in 921 bytes} [Tue Jul 26 15:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:05:26.414488 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5046136-e8a6-4eb3-b09d-ca7e46c4e12a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:26.414759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5046136-e8a6-4eb3-b09d-ca7e46c4e12a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:26.414990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5046136-e8a6-4eb3-b09d-ca7e46c4e12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:26.456974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-a5046136-e8a6-4eb3-b09d-ca7e46c4e12a None None] MFA Rules not processed for user `6a4278fe995e432d88f4346d6f54218b`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:26.458439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a5046136-e8a6-4eb3-b09d-ca7e46c4e12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:26.655445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 870/1758] 10.222.0.94 () {58 vars in 1018 bytes} [Tue Jul 26 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:26.700735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f129b-7b86-412e-abd5-15d3a45495d3 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:26.700735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-886f129b-7b86-412e-abd5-15d3a45495d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:26.766993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f129b-7b86-412e-abd5-15d3a45495d3 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:26.780304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd1989-1bff-4676-8229-660698fa95e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:26.780943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd1989-1bff-4676-8229-660698fa95e4 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:26.781217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd1989-1bff-4676-8229-660698fa95e4 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:26.781487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd1989-1bff-4676-8229-660698fa95e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:26.823755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd1989-1bff-4676-8229-660698fa95e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:26.824594 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd1989-1bff-4676-8229-660698fa95e4 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:26.876006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 889/1759] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:27.349325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5046136-e8a6-4eb3-b09d-ca7e46c4e12a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:27.351006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a5046136-e8a6-4eb3-b09d-ca7e46c4e12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:27.526822 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5046136-e8a6-4eb3-b09d-ca7e46c4e12a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:27.550924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa5af1-093a-405a-b049-8ea1e41aa8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:27.553016 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa5af1-093a-405a-b049-8ea1e41aa8a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:27.553781 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa5af1-093a-405a-b049-8ea1e41aa8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:27.554450 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa5af1-093a-405a-b049-8ea1e41aa8a2 admin admin] PATH_INFO: `/v3/projects/3d04940b01354bcabce0a75683732b0d` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:27.559398 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa5af1-093a-405a-b049-8ea1e41aa8a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d04940b01354bcabce0a75683732b0d)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:27.580729 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa5af1-093a-405a-b049-8ea1e41aa8a2 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:27.912926 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:27.940913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-fafa5af1-093a-405a-b049-8ea1e41aa8a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d04940b01354bcabce0a75683732b0d', 'request_id': 'req-fafa5af1-093a-405a-b049-8ea1e41aa8a2'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:27.956786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-fafa5af1-093a-405a-b049-8ea1e41aa8a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d04940b01354bcabce0a75683732b0d', 'request_id': 'req-fafa5af1-093a-405a-b049-8ea1e41aa8a2'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:27.971907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-fafa5af1-093a-405a-b049-8ea1e41aa8a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d04940b01354bcabce0a75683732b0d', 'request_id': 'req-fafa5af1-093a-405a-b049-8ea1e41aa8a2'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:28.000881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:28.084714 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-fafa5af1-093a-405a-b049-8ea1e41aa8a2 admin admin] The token cache is being invalidate because project 3d04940b01354bcabce0a75683732b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:05:28.085121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-fafa5af1-093a-405a-b049-8ea1e41aa8a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:28.087963 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-fafa5af1-093a-405a-b049-8ea1e41aa8a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:28.093303 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 871/1760] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:05:27 2022] DELETE /identity/v3/projects/3d04940b01354bcabce0a75683732b0d => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:28.100349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd1989-1bff-4676-8229-660698fa95e4 service nova] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:28.133570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd1989-1bff-4676-8229-660698fa95e4 service nova] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:28.296146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0441f9-b1ef-44d6-a5c5-5b97e94c55aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:28.296835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0441f9-b1ef-44d6-a5c5-5b97e94c55aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:28.297121 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0441f9-b1ef-44d6-a5c5-5b97e94c55aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:28.297512 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0441f9-b1ef-44d6-a5c5-5b97e94c55aa admin admin] PATH_INFO: `/v3/users/16601214f4ec49c08798a1e30f65c804` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:28.305421 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0441f9-b1ef-44d6-a5c5-5b97e94c55aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=16601214f4ec49c08798a1e30f65c804)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:28.308662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0441f9-b1ef-44d6-a5c5-5b97e94c55aa admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:28.578606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1a0441f9-b1ef-44d6-a5c5-5b97e94c55aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16601214f4ec49c08798a1e30f65c804'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:28.589213 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1a0441f9-b1ef-44d6-a5c5-5b97e94c55aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16601214f4ec49c08798a1e30f65c804'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:28.722759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1a0441f9-b1ef-44d6-a5c5-5b97e94c55aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16601214f4ec49c08798a1e30f65c804'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:28.740290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1a0441f9-b1ef-44d6-a5c5-5b97e94c55aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16601214f4ec49c08798a1e30f65c804'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:28.741152 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1a0441f9-b1ef-44d6-a5c5-5b97e94c55aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16601214f4ec49c08798a1e30f65c804'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:28.749752 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 890/1761] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:05:28 2022] DELETE /identity/v3/users/16601214f4ec49c08798a1e30f65c804 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:29.106308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ac6ac-026f-4e1f-981d-1eb495b54174 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:29.106308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ac6ac-026f-4e1f-981d-1eb495b54174 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:29.106308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ac6ac-026f-4e1f-981d-1eb495b54174 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:29.109776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 872/1762] 10.222.0.94 () {54 vars in 921 bytes} [Tue Jul 26 15:05:29 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:05:29.119456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e591f-faa1-4a2c-8949-81335872ee85 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:29.120048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e591f-faa1-4a2c-8949-81335872ee85 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:29.121777 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e591f-faa1-4a2c-8949-81335872ee85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:29.166641 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-180e591f-faa1-4a2c-8949-81335872ee85 None None] MFA Rules not processed for user `6a4278fe995e432d88f4346d6f54218b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:29.167969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-180e591f-faa1-4a2c-8949-81335872ee85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:29.275850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 891/1763] 10.222.0.94 () {58 vars in 1018 bytes} [Tue Jul 26 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:29.339778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ac6ac-026f-4e1f-981d-1eb495b54174 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:29.345266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c30ac6ac-026f-4e1f-981d-1eb495b54174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:29.355816 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e591f-faa1-4a2c-8949-81335872ee85 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:29.357742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-180e591f-faa1-4a2c-8949-81335872ee85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:29.380495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e591f-faa1-4a2c-8949-81335872ee85 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:29.396521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23a18c-7a95-42b5-b498-917c3c3886cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:29.397395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23a18c-7a95-42b5-b498-917c3c3886cd service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:29.398327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23a18c-7a95-42b5-b498-917c3c3886cd service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:29.398817 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23a18c-7a95-42b5-b498-917c3c3886cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:29.414381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23a18c-7a95-42b5-b498-917c3c3886cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:29.417779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23a18c-7a95-42b5-b498-917c3c3886cd service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:29.437966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 892/1764] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:05:29.486428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ac6ac-026f-4e1f-981d-1eb495b54174 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:29.490814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54db03f-70be-484c-a152-92b2d35d65cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:29.491446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54db03f-70be-484c-a152-92b2d35d65cd service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:29.491759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54db03f-70be-484c-a152-92b2d35d65cd service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:29.492040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54db03f-70be-484c-a152-92b2d35d65cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:29.494525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c54db03f-70be-484c-a152-92b2d35d65cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:29.566118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54db03f-70be-484c-a152-92b2d35d65cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:29.567582 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54db03f-70be-484c-a152-92b2d35d65cd service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:29.578228 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 873/1765] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:05:29.716098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23a18c-7a95-42b5-b498-917c3c3886cd service nova] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:29.716098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-0c23a18c-7a95-42b5-b498-917c3c3886cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:29.792599 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23a18c-7a95-42b5-b498-917c3c3886cd service nova] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:29.797148 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe77b8-183d-4cbf-850e-b2e312aa8801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:29.797827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe77b8-183d-4cbf-850e-b2e312aa8801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:29.798146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe77b8-183d-4cbf-850e-b2e312aa8801 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:29.798416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe77b8-183d-4cbf-850e-b2e312aa8801 admin admin] PATH_INFO: `/v3/projects/75fa510331d3467d96dd647e7d65cf5e` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:29.799612 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe77b8-183d-4cbf-850e-b2e312aa8801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75fa510331d3467d96dd647e7d65cf5e)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:29.801509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe77b8-183d-4cbf-850e-b2e312aa8801 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:30.001314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a9fe77b8-183d-4cbf-850e-b2e312aa8801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75fa510331d3467d96dd647e7d65cf5e', 'request_id': 'req-a9fe77b8-183d-4cbf-850e-b2e312aa8801'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:30.007134 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a9fe77b8-183d-4cbf-850e-b2e312aa8801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75fa510331d3467d96dd647e7d65cf5e', 'request_id': 'req-a9fe77b8-183d-4cbf-850e-b2e312aa8801'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:30.119682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a9fe77b8-183d-4cbf-850e-b2e312aa8801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75fa510331d3467d96dd647e7d65cf5e', 'request_id': 'req-a9fe77b8-183d-4cbf-850e-b2e312aa8801'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:30.132187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a9fe77b8-183d-4cbf-850e-b2e312aa8801 admin admin] The token cache is being invalidate because project 75fa510331d3467d96dd647e7d65cf5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:05:30.132473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a9fe77b8-183d-4cbf-850e-b2e312aa8801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:30.133139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a9fe77b8-183d-4cbf-850e-b2e312aa8801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:30.135417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 893/1766] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:05:29 2022] DELETE /identity/v3/projects/75fa510331d3467d96dd647e7d65cf5e => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:30.143662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a062c-151d-49ac-849b-9787454c9a18 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:30.144300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a062c-151d-49ac-849b-9787454c9a18 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:30.144758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a062c-151d-49ac-849b-9787454c9a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:30.172916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-bc4a062c-151d-49ac-849b-9787454c9a18 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:30.174213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-bc4a062c-151d-49ac-849b-9787454c9a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:30.246746 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 874/1767] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:30.255318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a003a6-30bc-43e4-922d-e61a954d1491 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:30.255647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a003a6-30bc-43e4-922d-e61a954d1491 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:30.255921 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a003a6-30bc-43e4-922d-e61a954d1491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:30.278003 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-48a003a6-30bc-43e4-922d-e61a954d1491 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:30.279262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-48a003a6-30bc-43e4-922d-e61a954d1491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:30.287604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 894/1768] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:30.292848 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a062c-151d-49ac-849b-9787454c9a18 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:30.303992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a062c-151d-49ac-849b-9787454c9a18 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:30.309513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86cb30-2b22-4f2b-a69d-4494b6dd9e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:30.310287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86cb30-2b22-4f2b-a69d-4494b6dd9e0c admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:30.310572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86cb30-2b22-4f2b-a69d-4494b6dd9e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:30.310853 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86cb30-2b22-4f2b-a69d-4494b6dd9e0c admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:30.311632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86cb30-2b22-4f2b-a69d-4494b6dd9e0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:05:30.311951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86cb30-2b22-4f2b-a69d-4494b6dd9e0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:30.313611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86cb30-2b22-4f2b-a69d-4494b6dd9e0c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:30.332320 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 875/1769] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:05:30.337786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a003a6-30bc-43e4-922d-e61a954d1491 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:30.349787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a003a6-30bc-43e4-922d-e61a954d1491 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:30.356256 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821dcf84-6aa8-4dbe-8b9b-8e46b1fa9d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:30.357007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821dcf84-6aa8-4dbe-8b9b-8e46b1fa9d37 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:30.357381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821dcf84-6aa8-4dbe-8b9b-8e46b1fa9d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:30.357754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821dcf84-6aa8-4dbe-8b9b-8e46b1fa9d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:30.358765 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821dcf84-6aa8-4dbe-8b9b-8e46b1fa9d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:30.361046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821dcf84-6aa8-4dbe-8b9b-8e46b1fa9d37 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:30.521289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 895/1770] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:30 2022] POST /identity/v3/projects => generated 369 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:30.527290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86cb30-2b22-4f2b-a69d-4494b6dd9e0c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:30.587608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86cb30-2b22-4f2b-a69d-4494b6dd9e0c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:30.595306 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:30.703041 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602e9c4-8808-4d29-abb3-80c2fe626b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:30.704182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602e9c4-8808-4d29-abb3-80c2fe626b3a admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:30.704807 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602e9c4-8808-4d29-abb3-80c2fe626b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:30.705164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602e9c4-8808-4d29-abb3-80c2fe626b3a admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:30.706821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602e9c4-8808-4d29-abb3-80c2fe626b3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:30.710913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602e9c4-8808-4d29-abb3-80c2fe626b3a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:30.818366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 876/1771] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:30 2022] POST /identity/v3/users => generated 285 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:30.824006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821dcf84-6aa8-4dbe-8b9b-8e46b1fa9d37 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:30.843899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821dcf84-6aa8-4dbe-8b9b-8e46b1fa9d37 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:30.849770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8214d-a529-49a1-a765-b05606186556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:30.855719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8214d-a529-49a1-a765-b05606186556 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:30.855719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8214d-a529-49a1-a765-b05606186556 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:30.855719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8214d-a529-49a1-a765-b05606186556 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:30.855719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8214d-a529-49a1-a765-b05606186556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:30.855719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8214d-a529-49a1-a765-b05606186556 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:30.876297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 896/1772] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:30.880605 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602e9c4-8808-4d29-abb3-80c2fe626b3a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:30.893175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602e9c4-8808-4d29-abb3-80c2fe626b3a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:30.898883 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d304df-4bb5-4b07-8424-c83af440a723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:30.900616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d304df-4bb5-4b07-8424-c83af440a723 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:30.900919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d304df-4bb5-4b07-8424-c83af440a723 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:30.901201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d304df-4bb5-4b07-8424-c83af440a723 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:30.902035 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d304df-4bb5-4b07-8424-c83af440a723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:30.903769 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d304df-4bb5-4b07-8424-c83af440a723 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:30.925618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 877/1773] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:30.927645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8214d-a529-49a1-a765-b05606186556 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:30.960552 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8214d-a529-49a1-a765-b05606186556 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:30.967640 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4b56b-06e4-481b-8530-e0ca468e0ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:30.968438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4b56b-06e4-481b-8530-e0ca468e0ccd admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:30.968795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4b56b-06e4-481b-8530-e0ca468e0ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:30.969146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4b56b-06e4-481b-8530-e0ca468e0ccd admin admin] PATH_INFO: `/v3/projects/a38e3064609b45afa4a7fedbe946067c/users/5df481e2e1014a9590f23e6faedb42e2/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:30.988653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4b56b-06e4-481b-8530-e0ca468e0ccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a38e3064609b45afa4a7fedbe946067c, user_id=5df481e2e1014a9590f23e6faedb42e2, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:30.992933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4b56b-06e4-481b-8530-e0ca468e0ccd admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:31.126000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 897/1774] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:30 2022] PUT /identity/v3/projects/a38e3064609b45afa4a7fedbe946067c/users/5df481e2e1014a9590f23e6faedb42e2/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:31.139971 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5381f-f82d-45ab-9a71-636f0991ac12 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:31.140262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5381f-f82d-45ab-9a71-636f0991ac12 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:31.140549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5381f-f82d-45ab-9a71-636f0991ac12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:31.220338 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-0aa5381f-f82d-45ab-9a71-636f0991ac12 None None] MFA Rules not processed for user `5df481e2e1014a9590f23e6faedb42e2`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:31.224308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0aa5381f-f82d-45ab-9a71-636f0991ac12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:31.378746 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 878/1775] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:31 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:31.383449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4b56b-06e4-481b-8530-e0ca468e0ccd admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:31.504054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4b56b-06e4-481b-8530-e0ca468e0ccd admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:31.513633 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d672f99-fbe2-4b48-97ab-857d46c0cef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:31.515123 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d672f99-fbe2-4b48-97ab-857d46c0cef0 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:31.515895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d672f99-fbe2-4b48-97ab-857d46c0cef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:31.516666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d672f99-fbe2-4b48-97ab-857d46c0cef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:31.518194 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d672f99-fbe2-4b48-97ab-857d46c0cef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:31.522336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d672f99-fbe2-4b48-97ab-857d46c0cef0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:31.653053 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:31.662914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 898/1776] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:31 2022] POST /identity/v3/projects => generated 365 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:31.669980 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5381f-f82d-45ab-9a71-636f0991ac12 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:31.684873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5381f-f82d-45ab-9a71-636f0991ac12 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:31.884289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b0905-10ee-46d8-b375-c152fe421fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:31.888276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b0905-10ee-46d8-b375-c152fe421fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:31.888771 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b0905-10ee-46d8-b375-c152fe421fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:31.889066 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b0905-10ee-46d8-b375-c152fe421fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:31.889887 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b0905-10ee-46d8-b375-c152fe421fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:31.892310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b0905-10ee-46d8-b375-c152fe421fd8 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:32.028829 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 879/1777] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:31 2022] POST /identity/v3/users => generated 289 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:32.043946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d672f99-fbe2-4b48-97ab-857d46c0cef0 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:32.102343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d672f99-fbe2-4b48-97ab-857d46c0cef0 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:32.107928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7f42e-022d-49a9-9806-8abb12a24167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:32.109303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7f42e-022d-49a9-9806-8abb12a24167 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:32.110110 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7f42e-022d-49a9-9806-8abb12a24167 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:32.110765 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7f42e-022d-49a9-9806-8abb12a24167 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:32.112360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7f42e-022d-49a9-9806-8abb12a24167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:32.129415 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7f42e-022d-49a9-9806-8abb12a24167 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:32.184168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 899/1778] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:32.188581 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b0905-10ee-46d8-b375-c152fe421fd8 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:32.203599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b0905-10ee-46d8-b375-c152fe421fd8 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:32.233041 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449fed1-6275-43a2-8963-2b0486b7bd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:32.233751 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449fed1-6275-43a2-8963-2b0486b7bd79 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:32.234030 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449fed1-6275-43a2-8963-2b0486b7bd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:32.234393 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449fed1-6275-43a2-8963-2b0486b7bd79 admin admin] PATH_INFO: `/v3/projects/1419b9801dba4fbc96b44fea1502602b/users/2fd9c92da4f34404bf11f1cef41da721/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:32.313486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449fed1-6275-43a2-8963-2b0486b7bd79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1419b9801dba4fbc96b44fea1502602b, user_id=2fd9c92da4f34404bf11f1cef41da721, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:32.323740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449fed1-6275-43a2-8963-2b0486b7bd79 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:32.487701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 880/1779] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:32 2022] PUT /identity/v3/projects/1419b9801dba4fbc96b44fea1502602b/users/2fd9c92da4f34404bf11f1cef41da721/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:32.491790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7f42e-022d-49a9-9806-8abb12a24167 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:32.516083 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7f42e-022d-49a9-9806-8abb12a24167 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:32.661323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bef59a-c8d5-41da-bba4-c27cad6f3fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:32.665834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bef59a-c8d5-41da-bba4-c27cad6f3fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:32.665834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bef59a-c8d5-41da-bba4-c27cad6f3fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:32.665834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bef59a-c8d5-41da-bba4-c27cad6f3fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:32.672064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bef59a-c8d5-41da-bba4-c27cad6f3fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:32.675443 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bef59a-c8d5-41da-bba4-c27cad6f3fb0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:32.703552 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 900/1780] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:05:32.708251 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449fed1-6275-43a2-8963-2b0486b7bd79 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:32.733314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449fed1-6275-43a2-8963-2b0486b7bd79 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:32.764356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e03e89-2f47-4788-9d37-dbf103181644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:32.767207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03e89-2f47-4788-9d37-dbf103181644 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:32.767207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03e89-2f47-4788-9d37-dbf103181644 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:32.767207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03e89-2f47-4788-9d37-dbf103181644 admin admin] PATH_INFO: `/v3/projects/1419b9801dba4fbc96b44fea1502602b/users/2fd9c92da4f34404bf11f1cef41da721/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:32.775795 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e03e89-2f47-4788-9d37-dbf103181644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1419b9801dba4fbc96b44fea1502602b, user_id=2fd9c92da4f34404bf11f1cef41da721, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:32.784329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e03e89-2f47-4788-9d37-dbf103181644 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:32.826041 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 881/1781] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:32 2022] PUT /identity/v3/projects/1419b9801dba4fbc96b44fea1502602b/users/2fd9c92da4f34404bf11f1cef41da721/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:32.844031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6e2c2-c252-442b-a0c1-12bc851da034 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:32.844973 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6e2c2-c252-442b-a0c1-12bc851da034 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:32.845910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6e2c2-c252-442b-a0c1-12bc851da034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:32.946692 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-4de6e2c2-c252-442b-a0c1-12bc851da034 None None] MFA Rules not processed for user `2fd9c92da4f34404bf11f1cef41da721`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:32.948415 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4de6e2c2-c252-442b-a0c1-12bc851da034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:33.175433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 901/1782] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:05:33.208009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e03e89-2f47-4788-9d37-dbf103181644 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:33.210134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f1e03e89-2f47-4788-9d37-dbf103181644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:33.367273 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e03e89-2f47-4788-9d37-dbf103181644 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:33.373628 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c309b6-1880-45f3-a190-e23f46fbcfd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:33.374315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c309b6-1880-45f3-a190-e23f46fbcfd4 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:33.375040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c309b6-1880-45f3-a190-e23f46fbcfd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:33.375331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c309b6-1880-45f3-a190-e23f46fbcfd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:33.393759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c309b6-1880-45f3-a190-e23f46fbcfd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:33.395386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c309b6-1880-45f3-a190-e23f46fbcfd4 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:33.409263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 882/1783] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:33.602347 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6e2c2-c252-442b-a0c1-12bc851da034 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:33.629308 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6e2c2-c252-442b-a0c1-12bc851da034 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:33.640134 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebaee31-952c-4855-9d63-57f648260f2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:33.641036 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebaee31-952c-4855-9d63-57f648260f2f service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:33.641970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebaee31-952c-4855-9d63-57f648260f2f service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:33.642444 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebaee31-952c-4855-9d63-57f648260f2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:33.772538 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebaee31-952c-4855-9d63-57f648260f2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:33.776075 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebaee31-952c-4855-9d63-57f648260f2f service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:33.786944 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 902/1784] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:33 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:34.537666 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:34.730791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:35.616758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c309b6-1880-45f3-a190-e23f46fbcfd4 service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:35.618989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a4c309b6-1880-45f3-a190-e23f46fbcfd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:35.780955 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c309b6-1880-45f3-a190-e23f46fbcfd4 service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:35.788901 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb1c1b-e607-45db-9bd3-257e2056c0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:35.790064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb1c1b-e607-45db-9bd3-257e2056c0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:35.790567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb1c1b-e607-45db-9bd3-257e2056c0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:35.791034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb1c1b-e607-45db-9bd3-257e2056c0f5 admin admin] PATH_INFO: `/v3/users/65e69af7926a40788365dc6b194bc486` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:35.793420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb1c1b-e607-45db-9bd3-257e2056c0f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65e69af7926a40788365dc6b194bc486)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:35.797279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb1c1b-e607-45db-9bd3-257e2056c0f5 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:36.041986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e6bb1c1b-e607-45db-9bd3-257e2056c0f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65e69af7926a40788365dc6b194bc486'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:36.043250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e6bb1c1b-e607-45db-9bd3-257e2056c0f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65e69af7926a40788365dc6b194bc486'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:36.044100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e6bb1c1b-e607-45db-9bd3-257e2056c0f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65e69af7926a40788365dc6b194bc486'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:36.066055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e6bb1c1b-e607-45db-9bd3-257e2056c0f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65e69af7926a40788365dc6b194bc486'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:36.091701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:36.095128 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e6bb1c1b-e607-45db-9bd3-257e2056c0f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65e69af7926a40788365dc6b194bc486'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:36.257696 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 883/1785] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:05:35 2022] DELETE /identity/v3/users/65e69af7926a40788365dc6b194bc486 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:36.268634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332713a-12a9-4a31-9693-af9d68e2edcd None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:36.269237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332713a-12a9-4a31-9693-af9d68e2edcd None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:36.269595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332713a-12a9-4a31-9693-af9d68e2edcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:36.327386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b332713a-12a9-4a31-9693-af9d68e2edcd None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:36.329669 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b332713a-12a9-4a31-9693-af9d68e2edcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:36.463560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 903/1786] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:36.488006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb1c1b-e607-45db-9bd3-257e2056c0f5 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:36.490431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e6bb1c1b-e607-45db-9bd3-257e2056c0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:36.607187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:36.678020 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb1c1b-e607-45db-9bd3-257e2056c0f5 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:36.689767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef94f8e-5bca-45ed-8fea-d50c1a878d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:36.690726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef94f8e-5bca-45ed-8fea-d50c1a878d28 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:36.691169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef94f8e-5bca-45ed-8fea-d50c1a878d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:36.691663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef94f8e-5bca-45ed-8fea-d50c1a878d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:36.705174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef94f8e-5bca-45ed-8fea-d50c1a878d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:36.707019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef94f8e-5bca-45ed-8fea-d50c1a878d28 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:36.720188 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 884/1787] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:36.847247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332713a-12a9-4a31-9693-af9d68e2edcd None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:36.851352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b332713a-12a9-4a31-9693-af9d68e2edcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:37.111099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332713a-12a9-4a31-9693-af9d68e2edcd None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:37.122376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531f2e5-a404-4363-8050-baab72d23020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:37.123642 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531f2e5-a404-4363-8050-baab72d23020 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:37.131177 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531f2e5-a404-4363-8050-baab72d23020 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:37.131637 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531f2e5-a404-4363-8050-baab72d23020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:37.133979 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9531f2e5-a404-4363-8050-baab72d23020 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:37.332269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531f2e5-a404-4363-8050-baab72d23020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:37.340647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531f2e5-a404-4363-8050-baab72d23020 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:37.358214 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 904/1788] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:37.941069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:39.301875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef94f8e-5bca-45ed-8fea-d50c1a878d28 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:39.301875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-fef94f8e-5bca-45ed-8fea-d50c1a878d28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:39.336109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef94f8e-5bca-45ed-8fea-d50c1a878d28 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:39.350508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:39.360694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:39.360694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:39.360694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] PATH_INFO: `/v3/projects/301672d4250b44caa2a2442a8a79f72e` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:39.360694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=301672d4250b44caa2a2442a8a79f72e)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:39.364253 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:39.515673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '301672d4250b44caa2a2442a8a79f72e', 'request_id': 'req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:39.523176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '301672d4250b44caa2a2442a8a79f72e', 'request_id': 'req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:39.534335 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '301672d4250b44caa2a2442a8a79f72e', 'request_id': 'req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:39.718902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] The token cache is being invalidate because project 301672d4250b44caa2a2442a8a79f72e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:05:39.719271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:39.720085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:39.721727 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 885/1789] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:05:39 2022] DELETE /identity/v3/projects/301672d4250b44caa2a2442a8a79f72e => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:39.726739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531f2e5-a404-4363-8050-baab72d23020 service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:39.748311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531f2e5-a404-4363-8050-baab72d23020 service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:39.835376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e5f8f-feb7-46bf-beae-f887ac7ec403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:39.836231 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e5f8f-feb7-46bf-beae-f887ac7ec403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:39.837158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e5f8f-feb7-46bf-beae-f887ac7ec403 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:39.837538 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e5f8f-feb7-46bf-beae-f887ac7ec403 admin admin] PATH_INFO: `/v3/users/830b3eef9dec454fa62422e1c8486f2d` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:39.839214 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e5f8f-feb7-46bf-beae-f887ac7ec403 admin admin] RBAC: Authorizing `identity:delete_user(user_id=830b3eef9dec454fa62422e1c8486f2d)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:39.841528 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e5f8f-feb7-46bf-beae-f887ac7ec403 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:40.175922 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1e6e5f8f-feb7-46bf-beae-f887ac7ec403 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '830b3eef9dec454fa62422e1c8486f2d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:40.191765 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1e6e5f8f-feb7-46bf-beae-f887ac7ec403 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '830b3eef9dec454fa62422e1c8486f2d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:40.283001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1e6e5f8f-feb7-46bf-beae-f887ac7ec403 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '830b3eef9dec454fa62422e1c8486f2d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:40.301924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1e6e5f8f-feb7-46bf-beae-f887ac7ec403 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '830b3eef9dec454fa62422e1c8486f2d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:40.302835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1e6e5f8f-feb7-46bf-beae-f887ac7ec403 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '830b3eef9dec454fa62422e1c8486f2d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:40.309769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 905/1790] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:05:39 2022] DELETE /identity/v3/users/830b3eef9dec454fa62422e1c8486f2d => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:40.921123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:40.922734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:40.946897 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9e0e0-d1a1-4924-b69b-a59dc3828a2b admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:41.014913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e5f8f-feb7-46bf-beae-f887ac7ec403 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:41.016899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1e6e5f8f-feb7-46bf-beae-f887ac7ec403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:41.052441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:41.053171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:41.053461 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:41.053890 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] PATH_INFO: `/v3/projects/ce223560877b42b9b68a14f689ec43fe` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:41.055220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce223560877b42b9b68a14f689ec43fe)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:41.057518 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:41.076641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:41.100886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e5f8f-feb7-46bf-beae-f887ac7ec403 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:41.107045 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d241f8-8c4c-4a97-b126-e30dc1c8123c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:41.107748 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d241f8-8c4c-4a97-b126-e30dc1c8123c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:41.108059 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d241f8-8c4c-4a97-b126-e30dc1c8123c admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:41.108333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d241f8-8c4c-4a97-b126-e30dc1c8123c admin admin] PATH_INFO: `/v3/users/5df481e2e1014a9590f23e6faedb42e2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:41.109723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d241f8-8c4c-4a97-b126-e30dc1c8123c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5df481e2e1014a9590f23e6faedb42e2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:41.111802 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d241f8-8c4c-4a97-b126-e30dc1c8123c admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:41.114064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:41.269040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce223560877b42b9b68a14f689ec43fe', 'request_id': 'req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:41.277363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce223560877b42b9b68a14f689ec43fe', 'request_id': 'req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:41.284370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce223560877b42b9b68a14f689ec43fe', 'request_id': 'req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:41.432842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] The token cache is being invalidate because project ce223560877b42b9b68a14f689ec43fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:05:41.434041 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:41.435748 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:41.441037 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 886/1791] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:05:40 2022] DELETE /identity/v3/projects/ce223560877b42b9b68a14f689ec43fe => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:41.445181 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-12d241f8-8c4c-4a97-b126-e30dc1c8123c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5df481e2e1014a9590f23e6faedb42e2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:41.449001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:41.451853 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:41.460817 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-12d241f8-8c4c-4a97-b126-e30dc1c8123c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5df481e2e1014a9590f23e6faedb42e2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:41.576367 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f107d-bb5d-40df-81fb-c7e38f3ba12c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:41.581414 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3c174-6dd1-4489-bee3-95971e93c431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:41.582078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3c174-6dd1-4489-bee3-95971e93c431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:41.582366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3c174-6dd1-4489-bee3-95971e93c431 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:41.582673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3c174-6dd1-4489-bee3-95971e93c431 admin admin] PATH_INFO: `/v3/users/870b500ece3d425185a8c8d28524a27a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:41.583925 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3c174-6dd1-4489-bee3-95971e93c431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=870b500ece3d425185a8c8d28524a27a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:41.585885 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3c174-6dd1-4489-bee3-95971e93c431 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:41.636134 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-12d241f8-8c4c-4a97-b126-e30dc1c8123c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5df481e2e1014a9590f23e6faedb42e2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:41.660959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-12d241f8-8c4c-4a97-b126-e30dc1c8123c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5df481e2e1014a9590f23e6faedb42e2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:41.662053 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-12d241f8-8c4c-4a97-b126-e30dc1c8123c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5df481e2e1014a9590f23e6faedb42e2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:41.669208 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 906/1792] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:05:41 2022] DELETE /identity/v3/users/5df481e2e1014a9590f23e6faedb42e2 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:41.674536 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567115f-215f-42fb-8128-c947f0aae712 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:41.674831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567115f-215f-42fb-8128-c947f0aae712 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:41.675109 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567115f-215f-42fb-8128-c947f0aae712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:41.710275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-9567115f-215f-42fb-8128-c947f0aae712 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:41.710275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9567115f-215f-42fb-8128-c947f0aae712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:41.769907 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 907/1793] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:05:41.781178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567115f-215f-42fb-8128-c947f0aae712 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:41.782759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9567115f-215f-42fb-8128-c947f0aae712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:41.856785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567115f-215f-42fb-8128-c947f0aae712 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:41.861324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633db9b-1682-4df9-a868-f0dc509ccdcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:41.862099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633db9b-1682-4df9-a868-f0dc509ccdcc service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:41.862393 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633db9b-1682-4df9-a868-f0dc509ccdcc service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:41.862715 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633db9b-1682-4df9-a868-f0dc509ccdcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:41.877286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633db9b-1682-4df9-a868-f0dc509ccdcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:41.879578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633db9b-1682-4df9-a868-f0dc509ccdcc service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:41.897481 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-24b3c174-6dd1-4489-bee3-95971e93c431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '870b500ece3d425185a8c8d28524a27a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:41.898839 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-24b3c174-6dd1-4489-bee3-95971e93c431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '870b500ece3d425185a8c8d28524a27a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:41.901147 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-24b3c174-6dd1-4489-bee3-95971e93c431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '870b500ece3d425185a8c8d28524a27a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:41.907540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 908/1794] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:05:41.918012 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-24b3c174-6dd1-4489-bee3-95971e93c431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '870b500ece3d425185a8c8d28524a27a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:41.941451 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-24b3c174-6dd1-4489-bee3-95971e93c431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '870b500ece3d425185a8c8d28524a27a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:42.058501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 887/1795] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:05:41 2022] DELETE /identity/v3/users/870b500ece3d425185a8c8d28524a27a => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:42.627891 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633db9b-1682-4df9-a868-f0dc509ccdcc service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:42.629601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2633db9b-1682-4df9-a868-f0dc509ccdcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:42.644133 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633db9b-1682-4df9-a868-f0dc509ccdcc service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:42.759556 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:42.760917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:42.761542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:42.762111 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] PATH_INFO: `/v3/projects/a38e3064609b45afa4a7fedbe946067c` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:42.765980 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a38e3064609b45afa4a7fedbe946067c)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:42.771507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:42.957484 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3c174-6dd1-4489-bee3-95971e93c431 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:42.959954 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-24b3c174-6dd1-4489-bee3-95971e93c431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:42.983672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3c174-6dd1-4489-bee3-95971e93c431 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:42.985039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a38e3064609b45afa4a7fedbe946067c', 'request_id': 'req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:42.994603 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a38e3064609b45afa4a7fedbe946067c', 'request_id': 'req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:43.061339 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e8ab2-b095-44f9-b01e-5e944fbb67bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:43.062078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e8ab2-b095-44f9-b01e-5e944fbb67bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:43.062372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e8ab2-b095-44f9-b01e-5e944fbb67bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:43.062653 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e8ab2-b095-44f9-b01e-5e944fbb67bb admin admin] PATH_INFO: `/v3/projects/6b715f9142654bcabcda78a614862896` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:43.064084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e8ab2-b095-44f9-b01e-5e944fbb67bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b715f9142654bcabcda78a614862896)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:43.066199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e8ab2-b095-44f9-b01e-5e944fbb67bb admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:43.113475 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a38e3064609b45afa4a7fedbe946067c', 'request_id': 'req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:43.128577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] The token cache is being invalidate because project a38e3064609b45afa4a7fedbe946067c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:05:43.129024 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:43.130140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:43.132894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 909/1796] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:05:42 2022] DELETE /identity/v3/projects/a38e3064609b45afa4a7fedbe946067c => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:43.137777 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:43.139324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:43.156755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a4cfd-5a92-46f7-9d5c-06fc4c7b1c08 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:43.163597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620d253-db47-40fc-a535-03074cc49db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:43.165326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620d253-db47-40fc-a535-03074cc49db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:43.165691 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620d253-db47-40fc-a535-03074cc49db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:43.166064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620d253-db47-40fc-a535-03074cc49db8 admin admin] PATH_INFO: `/v3/users/2fd9c92da4f34404bf11f1cef41da721` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:43.167804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620d253-db47-40fc-a535-03074cc49db8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd9c92da4f34404bf11f1cef41da721)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:43.170182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620d253-db47-40fc-a535-03074cc49db8 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:43.232732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-418e8ab2-b095-44f9-b01e-5e944fbb67bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b715f9142654bcabcda78a614862896', 'request_id': 'req-418e8ab2-b095-44f9-b01e-5e944fbb67bb'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:43.240023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-418e8ab2-b095-44f9-b01e-5e944fbb67bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b715f9142654bcabcda78a614862896', 'request_id': 'req-418e8ab2-b095-44f9-b01e-5e944fbb67bb'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:43.249433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-418e8ab2-b095-44f9-b01e-5e944fbb67bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b715f9142654bcabcda78a614862896', 'request_id': 'req-418e8ab2-b095-44f9-b01e-5e944fbb67bb'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:43.567329 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7620d253-db47-40fc-a535-03074cc49db8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd9c92da4f34404bf11f1cef41da721'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:43.575900 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7620d253-db47-40fc-a535-03074cc49db8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd9c92da4f34404bf11f1cef41da721'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:44.016140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-418e8ab2-b095-44f9-b01e-5e944fbb67bb admin admin] The token cache is being invalidate because project 6b715f9142654bcabcda78a614862896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:05:44.016721 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-418e8ab2-b095-44f9-b01e-5e944fbb67bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:44.017424 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-418e8ab2-b095-44f9-b01e-5e944fbb67bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:44.019364 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 888/1797] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:05:42 2022] DELETE /identity/v3/projects/6b715f9142654bcabcda78a614862896 => generated 0 bytes in 1063 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:05:44.026212 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ea72b-b645-4dec-b5da-f45ec27ebf5b None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:44.026647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ea72b-b645-4dec-b5da-f45ec27ebf5b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:44.027441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ea72b-b645-4dec-b5da-f45ec27ebf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:44.053799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-165ea72b-b645-4dec-b5da-f45ec27ebf5b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:44.105142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-165ea72b-b645-4dec-b5da-f45ec27ebf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:44.173930 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 889/1798] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:44.189787 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44482eb-9a9c-471a-ab89-b1c3d587f9ad None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:44.190355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44482eb-9a9c-471a-ab89-b1c3d587f9ad None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:44.190970 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44482eb-9a9c-471a-ab89-b1c3d587f9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:44.226282 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-d44482eb-9a9c-471a-ab89-b1c3d587f9ad None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:44.228079 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d44482eb-9a9c-471a-ab89-b1c3d587f9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:44.237824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 890/1799] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:44.244045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44482eb-9a9c-471a-ab89-b1c3d587f9ad None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:44.280754 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44482eb-9a9c-471a-ab89-b1c3d587f9ad None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:44.288065 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ec0b7-a00a-46a5-9b5a-536af7059767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:44.289059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ec0b7-a00a-46a5-9b5a-536af7059767 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:44.289637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ec0b7-a00a-46a5-9b5a-536af7059767 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:44.290121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ec0b7-a00a-46a5-9b5a-536af7059767 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:44.291364 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ec0b7-a00a-46a5-9b5a-536af7059767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:05:44.291910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ec0b7-a00a-46a5-9b5a-536af7059767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:44.294837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ec0b7-a00a-46a5-9b5a-536af7059767 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:44.330961 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 891/1800] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:05:44.335779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ec0b7-a00a-46a5-9b5a-536af7059767 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:44.363204 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ec0b7-a00a-46a5-9b5a-536af7059767 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:44.370534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e2906-1b80-462b-ab6a-218989d7af02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:44.371276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e2906-1b80-462b-ab6a-218989d7af02 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:44.371659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e2906-1b80-462b-ab6a-218989d7af02 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:44.372009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e2906-1b80-462b-ab6a-218989d7af02 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:44.378446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e2906-1b80-462b-ab6a-218989d7af02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:44.395219 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e2906-1b80-462b-ab6a-218989d7af02 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:45.037262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7620d253-db47-40fc-a535-03074cc49db8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd9c92da4f34404bf11f1cef41da721'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:45.049906 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 892/1801] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:44 2022] POST /identity/v3/projects => generated 389 bytes in 715 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:05:45.057717 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e2906-1b80-462b-ab6a-218989d7af02 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:45.066698 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7620d253-db47-40fc-a535-03074cc49db8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd9c92da4f34404bf11f1cef41da721'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:45.077919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7620d253-db47-40fc-a535-03074cc49db8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd9c92da4f34404bf11f1cef41da721'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:45.090292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 910/1802] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:05:43 2022] DELETE /identity/v3/users/2fd9c92da4f34404bf11f1cef41da721 => generated 0 bytes in 1952 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:05:45.210481 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e2906-1b80-462b-ab6a-218989d7af02 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:45.350254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d895e3-208a-406c-aabf-3645fb5f24d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:45.351019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d895e3-208a-406c-aabf-3645fb5f24d1 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:45.351370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d895e3-208a-406c-aabf-3645fb5f24d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:45.352268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d895e3-208a-406c-aabf-3645fb5f24d1 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:45.353725 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d895e3-208a-406c-aabf-3645fb5f24d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:45.355904 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d895e3-208a-406c-aabf-3645fb5f24d1 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:45.484024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 893/1803] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:45 2022] POST /identity/v3/users => generated 295 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:05:45.485431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620d253-db47-40fc-a535-03074cc49db8 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:45.488006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7620d253-db47-40fc-a535-03074cc49db8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:45.510757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:45.518182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620d253-db47-40fc-a535-03074cc49db8 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:45.528839 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7502d66-8d6f-482e-a1fd-ffb99733782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:45.530801 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7502d66-8d6f-482e-a1fd-ffb99733782e admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:45.530801 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7502d66-8d6f-482e-a1fd-ffb99733782e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:45.530801 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7502d66-8d6f-482e-a1fd-ffb99733782e admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:45.531307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7502d66-8d6f-482e-a1fd-ffb99733782e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:45.533259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7502d66-8d6f-482e-a1fd-ffb99733782e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:45.545759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 911/1804] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:45.552040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d895e3-208a-406c-aabf-3645fb5f24d1 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:45.564848 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d895e3-208a-406c-aabf-3645fb5f24d1 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:45.572544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d22e5-b680-4286-8c4e-278c190e1039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:45.573598 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d22e5-b680-4286-8c4e-278c190e1039 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:45.574154 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d22e5-b680-4286-8c4e-278c190e1039 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:45.574567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d22e5-b680-4286-8c4e-278c190e1039 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:45.575559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d22e5-b680-4286-8c4e-278c190e1039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:45.577329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d22e5-b680-4286-8c4e-278c190e1039 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:45.590201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 894/1805] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:45.596425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7502d66-8d6f-482e-a1fd-ffb99733782e admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:45.607923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7502d66-8d6f-482e-a1fd-ffb99733782e admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:45.613671 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470a9c7-45b7-479f-8ca8-75c66bd4191a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:45.615118 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470a9c7-45b7-479f-8ca8-75c66bd4191a admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:45.615647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470a9c7-45b7-479f-8ca8-75c66bd4191a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:45.616105 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470a9c7-45b7-479f-8ca8-75c66bd4191a admin admin] PATH_INFO: `/v3/projects/1bfd7adf45434acc8473bd7b7b1a5fcb/users/45765858d2cb4bbfa0ba15947b814a7d/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:45.637777 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470a9c7-45b7-479f-8ca8-75c66bd4191a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bfd7adf45434acc8473bd7b7b1a5fcb, user_id=45765858d2cb4bbfa0ba15947b814a7d, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:45.641922 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470a9c7-45b7-479f-8ca8-75c66bd4191a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:45.799372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d22e5-b680-4286-8c4e-278c190e1039 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:45.800071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 912/1806] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:45 2022] PUT /identity/v3/projects/1bfd7adf45434acc8473bd7b7b1a5fcb/users/45765858d2cb4bbfa0ba15947b814a7d/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:45.812392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-638d22e5-b680-4286-8c4e-278c190e1039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:45.813345 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01b9cc-434c-4a73-90e7-418c8862b8dd None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:45.814995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01b9cc-434c-4a73-90e7-418c8862b8dd None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:45.815761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01b9cc-434c-4a73-90e7-418c8862b8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:45.827229 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d22e5-b680-4286-8c4e-278c190e1039 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:45.889407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-fd01b9cc-434c-4a73-90e7-418c8862b8dd None None] MFA Rules not processed for user `45765858d2cb4bbfa0ba15947b814a7d`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:45.892281 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-fd01b9cc-434c-4a73-90e7-418c8862b8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:45.957210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e59d70-ced9-4d09-8469-d2bb069275a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:45.964558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e59d70-ced9-4d09-8469-d2bb069275a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:45.964927 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e59d70-ced9-4d09-8469-d2bb069275a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:45.965220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e59d70-ced9-4d09-8469-d2bb069275a6 admin admin] PATH_INFO: `/v3/projects/1419b9801dba4fbc96b44fea1502602b` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:45.966649 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e59d70-ced9-4d09-8469-d2bb069275a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1419b9801dba4fbc96b44fea1502602b)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:45.968800 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e59d70-ced9-4d09-8469-d2bb069275a6 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:46.019788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:46.024964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 913/1807] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:46.030381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01b9cc-434c-4a73-90e7-418c8862b8dd None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:46.048490 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01b9cc-434c-4a73-90e7-418c8862b8dd None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:46.054413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162129f5-dc85-4b8f-a7e7-19548821c360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:46.055128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162129f5-dc85-4b8f-a7e7-19548821c360 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:46.055976 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162129f5-dc85-4b8f-a7e7-19548821c360 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:46.056441 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162129f5-dc85-4b8f-a7e7-19548821c360 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:46.057343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162129f5-dc85-4b8f-a7e7-19548821c360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:46.059275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162129f5-dc85-4b8f-a7e7-19548821c360 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:46.169758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-31e59d70-ced9-4d09-8469-d2bb069275a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1419b9801dba4fbc96b44fea1502602b', 'request_id': 'req-31e59d70-ced9-4d09-8469-d2bb069275a6'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:46.184097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-31e59d70-ced9-4d09-8469-d2bb069275a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1419b9801dba4fbc96b44fea1502602b', 'request_id': 'req-31e59d70-ced9-4d09-8469-d2bb069275a6'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:46.194692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-31e59d70-ced9-4d09-8469-d2bb069275a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1419b9801dba4fbc96b44fea1502602b', 'request_id': 'req-31e59d70-ced9-4d09-8469-d2bb069275a6'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:46.213391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 914/1808] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:46 2022] POST /identity/v3/projects => generated 391 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:05:46.223343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162129f5-dc85-4b8f-a7e7-19548821c360 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:46.331836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162129f5-dc85-4b8f-a7e7-19548821c360 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:46.332005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4b697-def5-4a04-97b6-d38f237a1322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:46.332552 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4b697-def5-4a04-97b6-d38f237a1322 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:46.332843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4b697-def5-4a04-97b6-d38f237a1322 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:46.333120 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4b697-def5-4a04-97b6-d38f237a1322 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:46.333967 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4b697-def5-4a04-97b6-d38f237a1322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:46.335931 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4b697-def5-4a04-97b6-d38f237a1322 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:46.368923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-31e59d70-ced9-4d09-8469-d2bb069275a6 admin admin] The token cache is being invalidate because project 1419b9801dba4fbc96b44fea1502602b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:05:46.369221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-31e59d70-ced9-4d09-8469-d2bb069275a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:46.369899 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-31e59d70-ced9-4d09-8469-d2bb069275a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:46.371912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 895/1809] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:05:45 2022] DELETE /identity/v3/projects/1419b9801dba4fbc96b44fea1502602b => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:05:46.377817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3e57c-e14e-4a42-8625-af2c8c0906fc None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:46.378088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3e57c-e14e-4a42-8625-af2c8c0906fc None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:46.378371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3e57c-e14e-4a42-8625-af2c8c0906fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:46.401452 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-07e3e57c-e14e-4a42-8625-af2c8c0906fc None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:46.402710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-07e3e57c-e14e-4a42-8625-af2c8c0906fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:46.412433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 896/1810] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:46.434665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdc01e-ab28-4ed2-b6cc-1663d0c640b5 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:46.434993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdc01e-ab28-4ed2-b6cc-1663d0c640b5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:46.434993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdc01e-ab28-4ed2-b6cc-1663d0c640b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:46.466297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-bffdc01e-ab28-4ed2-b6cc-1663d0c640b5 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:46.467766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-bffdc01e-ab28-4ed2-b6cc-1663d0c640b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:46.483566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 897/1811] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:46.488278 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdc01e-ab28-4ed2-b6cc-1663d0c640b5 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:46.502732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdc01e-ab28-4ed2-b6cc-1663d0c640b5 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:46.519671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020711c-0621-4c90-9acd-018f42e9a022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:46.520834 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020711c-0621-4c90-9acd-018f42e9a022 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:46.521509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020711c-0621-4c90-9acd-018f42e9a022 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:46.521897 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020711c-0621-4c90-9acd-018f42e9a022 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:46.523257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020711c-0621-4c90-9acd-018f42e9a022 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:05:46.523954 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020711c-0621-4c90-9acd-018f42e9a022 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:46.527545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020711c-0621-4c90-9acd-018f42e9a022 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:46.527774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 915/1812] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:46 2022] POST /identity/v3/users => generated 302 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:46.533159 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4b697-def5-4a04-97b6-d38f237a1322 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:46.537116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-79c4b697-def5-4a04-97b6-d38f237a1322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:46.574089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4b697-def5-4a04-97b6-d38f237a1322 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:46.577378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 898/1813] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:05:46.578902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020711c-0621-4c90-9acd-018f42e9a022 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:46.589693 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ac083-30f2-444d-b408-2bb09bc49dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:46.592074 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ac083-30f2-444d-b408-2bb09bc49dcf admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:46.592938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ac083-30f2-444d-b408-2bb09bc49dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:46.593051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020711c-0621-4c90-9acd-018f42e9a022 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:46.593478 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ac083-30f2-444d-b408-2bb09bc49dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:46.594852 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ac083-30f2-444d-b408-2bb09bc49dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:46.600504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ac083-30f2-444d-b408-2bb09bc49dcf admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:46.619271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797eff02-0d35-4cdd-86b3-4da2f9cf0140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:46.620133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797eff02-0d35-4cdd-86b3-4da2f9cf0140 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:46.620495 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797eff02-0d35-4cdd-86b3-4da2f9cf0140 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:46.620870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797eff02-0d35-4cdd-86b3-4da2f9cf0140 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:46.621803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797eff02-0d35-4cdd-86b3-4da2f9cf0140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:46.631003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797eff02-0d35-4cdd-86b3-4da2f9cf0140 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:46.631336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 916/1814] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:46.634105 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ac083-30f2-444d-b408-2bb09bc49dcf admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:46.654244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ac083-30f2-444d-b408-2bb09bc49dcf admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:46.663149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7412111-9c03-4a34-910f-b06b7a67b8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:46.666092 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7412111-9c03-4a34-910f-b06b7a67b8aa admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:46.666594 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7412111-9c03-4a34-910f-b06b7a67b8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:46.667072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7412111-9c03-4a34-910f-b06b7a67b8aa admin admin] PATH_INFO: `/v3/projects/e49eef18ae0b4e31838c4e66ab9fd999/users/0228360f03a04e0fb4201f58d01e47f4/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:46.705314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7412111-9c03-4a34-910f-b06b7a67b8aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e49eef18ae0b4e31838c4e66ab9fd999, user_id=0228360f03a04e0fb4201f58d01e47f4, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:46.710506 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7412111-9c03-4a34-910f-b06b7a67b8aa admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:46.801296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 899/1815] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:46 2022] POST /identity/v3/projects => generated 385 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:46.802737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797eff02-0d35-4cdd-86b3-4da2f9cf0140 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:46.825644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 917/1816] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:46 2022] PUT /identity/v3/projects/e49eef18ae0b4e31838c4e66ab9fd999/users/0228360f03a04e0fb4201f58d01e47f4/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:46.835364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7412111-9c03-4a34-910f-b06b7a67b8aa admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:46.957217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7412111-9c03-4a34-910f-b06b7a67b8aa admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:46.976625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8a014-c3e7-44b1-9b5a-2209e3c5a72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:46.977918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8a014-c3e7-44b1-9b5a-2209e3c5a72d admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:46.981455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8a014-c3e7-44b1-9b5a-2209e3c5a72d admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:46.981964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8a014-c3e7-44b1-9b5a-2209e3c5a72d admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:46.983308 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8a014-c3e7-44b1-9b5a-2209e3c5a72d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:46.986506 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8a014-c3e7-44b1-9b5a-2209e3c5a72d admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:47.021781 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797eff02-0d35-4cdd-86b3-4da2f9cf0140 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:47.024809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 918/1817] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:47.044154 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb00da9-5cb9-424e-88c8-0a0437ae28cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:47.045910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8a014-c3e7-44b1-9b5a-2209e3c5a72d admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:47.046073 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb00da9-5cb9-424e-88c8-0a0437ae28cf admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:47.046250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb00da9-5cb9-424e-88c8-0a0437ae28cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:47.047753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb00da9-5cb9-424e-88c8-0a0437ae28cf admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:47.049558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb00da9-5cb9-424e-88c8-0a0437ae28cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:47.052985 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb00da9-5cb9-424e-88c8-0a0437ae28cf admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:47.091892 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8a014-c3e7-44b1-9b5a-2209e3c5a72d admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:47.108356 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923ef35-f9b1-4bde-97e8-dba061fd4f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:47.109201 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923ef35-f9b1-4bde-97e8-dba061fd4f88 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:47.109656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923ef35-f9b1-4bde-97e8-dba061fd4f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:47.110062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923ef35-f9b1-4bde-97e8-dba061fd4f88 admin admin] PATH_INFO: `/v3/projects/e49eef18ae0b4e31838c4e66ab9fd999/users/0228360f03a04e0fb4201f58d01e47f4/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:47.116544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2923ef35-f9b1-4bde-97e8-dba061fd4f88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e49eef18ae0b4e31838c4e66ab9fd999, user_id=0228360f03a04e0fb4201f58d01e47f4, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:47.126582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2923ef35-f9b1-4bde-97e8-dba061fd4f88 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:47.178533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 919/1818] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:47 2022] PUT /identity/v3/projects/e49eef18ae0b4e31838c4e66ab9fd999/users/0228360f03a04e0fb4201f58d01e47f4/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:47.178533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24edeb-c065-49e9-b230-d776b0d6dda3 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:47.178533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24edeb-c065-49e9-b230-d776b0d6dda3 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:47.178533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24edeb-c065-49e9-b230-d776b0d6dda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:47.236357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-9f24edeb-c065-49e9-b230-d776b0d6dda3 None None] MFA Rules not processed for user `0228360f03a04e0fb4201f58d01e47f4`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:47.236780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 900/1819] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:46 2022] POST /identity/v3/users => generated 293 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:47.238727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9f24edeb-c065-49e9-b230-d776b0d6dda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:47.241268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb00da9-5cb9-424e-88c8-0a0437ae28cf admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:47.271645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb00da9-5cb9-424e-88c8-0a0437ae28cf admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:47.390341 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 920/1820] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:47.407234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24edeb-c065-49e9-b230-d776b0d6dda3 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:47.409139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa0935-0d5c-40ce-a3ae-fc91edd14d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:47.409809 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa0935-0d5c-40ce-a3ae-fc91edd14d9d admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:47.410116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa0935-0d5c-40ce-a3ae-fc91edd14d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:47.410287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9f24edeb-c065-49e9-b230-d776b0d6dda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:47.410400 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa0935-0d5c-40ce-a3ae-fc91edd14d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:47.411070 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa0935-0d5c-40ce-a3ae-fc91edd14d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:47.412737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa0935-0d5c-40ce-a3ae-fc91edd14d9d admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:47.426478 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 901/1821] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:05:47.431910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa0935-0d5c-40ce-a3ae-fc91edd14d9d admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:47.443107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa0935-0d5c-40ce-a3ae-fc91edd14d9d admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:47.453415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b784c-46a8-41cf-acc0-fdeaa13557d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:47.454737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b784c-46a8-41cf-acc0-fdeaa13557d2 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:47.455249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b784c-46a8-41cf-acc0-fdeaa13557d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:47.455762 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b784c-46a8-41cf-acc0-fdeaa13557d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:47.456951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b784c-46a8-41cf-acc0-fdeaa13557d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:47.458702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b784c-46a8-41cf-acc0-fdeaa13557d2 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:47.477863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 902/1822] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:05:47.482086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b784c-46a8-41cf-acc0-fdeaa13557d2 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:47.493426 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24edeb-c065-49e9-b230-d776b0d6dda3 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:47.499063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b784c-46a8-41cf-acc0-fdeaa13557d2 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:47.510102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5100b7-e767-44ab-8d8e-e995951fc18c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:47.510861 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5100b7-e767-44ab-8d8e-e995951fc18c service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:47.511169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5100b7-e767-44ab-8d8e-e995951fc18c service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:47.511407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2b3d4-cefc-460c-af6e-9cc746e2d1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:47.511575 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5100b7-e767-44ab-8d8e-e995951fc18c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:47.527405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2b3d4-cefc-460c-af6e-9cc746e2d1fb admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:47.527405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2b3d4-cefc-460c-af6e-9cc746e2d1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:47.527405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2b3d4-cefc-460c-af6e-9cc746e2d1fb admin admin] PATH_INFO: `/v3/projects/e57f3bc37fcb46699068dc3a0f880e05/users/be1bf818dd6d4b62a7ecf70ac3fec71d/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:47.527737 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5100b7-e767-44ab-8d8e-e995951fc18c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:47.528979 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5100b7-e767-44ab-8d8e-e995951fc18c service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:47.532134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f2b3d4-cefc-460c-af6e-9cc746e2d1fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e57f3bc37fcb46699068dc3a0f880e05, user_id=be1bf818dd6d4b62a7ecf70ac3fec71d, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:47.535009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f2b3d4-cefc-460c-af6e-9cc746e2d1fb admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:47.545280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 921/1823] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:47.547789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f30aa-82bc-412a-b7f7-fc27531ee2a8 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:47.548361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f30aa-82bc-412a-b7f7-fc27531ee2a8 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:47.548981 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f30aa-82bc-412a-b7f7-fc27531ee2a8 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:47.551678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 922/1824] 10.222.0.94 () {54 vars in 921 bytes} [Tue Jul 26 15:05:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:05:47.563272 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23bc8e-a627-49f5-afcf-44009288927e None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:47.563624 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23bc8e-a627-49f5-afcf-44009288927e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:47.564133 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23bc8e-a627-49f5-afcf-44009288927e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:47.593721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-cd23bc8e-a627-49f5-afcf-44009288927e None None] MFA Rules not processed for user `6a4278fe995e432d88f4346d6f54218b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:47.595002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-cd23bc8e-a627-49f5-afcf-44009288927e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:47.650433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 903/1825] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:47 2022] PUT /identity/v3/projects/e57f3bc37fcb46699068dc3a0f880e05/users/be1bf818dd6d4b62a7ecf70ac3fec71d/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:47.652012 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2b3d4-cefc-460c-af6e-9cc746e2d1fb admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:47.655665 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d9f2b3d4-cefc-460c-af6e-9cc746e2d1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:47.708247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 923/1826] 10.222.0.94 () {58 vars in 1018 bytes} [Tue Jul 26 15:05:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:05:47.710385 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9da8-8ac1-4302-bacf-4bedae029d63 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:47.712143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9da8-8ac1-4302-bacf-4bedae029d63 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:47.712589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9da8-8ac1-4302-bacf-4bedae029d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:47.766579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-29cb9da8-8ac1-4302-bacf-4bedae029d63 None None] MFA Rules not processed for user `be1bf818dd6d4b62a7ecf70ac3fec71d`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:47.768197 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-29cb9da8-8ac1-4302-bacf-4bedae029d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:47.768983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f2b3d4-cefc-460c-af6e-9cc746e2d1fb admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:47.777794 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f81e15-2ef4-4ec0-acb6-7a3a35e2748f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:47.779711 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f81e15-2ef4-4ec0-acb6-7a3a35e2748f service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:47.780467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f81e15-2ef4-4ec0-acb6-7a3a35e2748f service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:47.781105 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f81e15-2ef4-4ec0-acb6-7a3a35e2748f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:47.784792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c6f81e15-2ef4-4ec0-acb6-7a3a35e2748f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:47.872042 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 924/1827] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:47.873661 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb9da8-8ac1-4302-bacf-4bedae029d63 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:47.876332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-29cb9da8-8ac1-4302-bacf-4bedae029d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:47.916855 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f81e15-2ef4-4ec0-acb6-7a3a35e2748f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:47.918391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f81e15-2ef4-4ec0-acb6-7a3a35e2748f service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:47.932786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 904/1828] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:47.933866 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f81e15-2ef4-4ec0-acb6-7a3a35e2748f service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:48.030653 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f81e15-2ef4-4ec0-acb6-7a3a35e2748f service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:48.039075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1c8ba-5e1f-4c2a-8b0f-b22d8a7093dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:48.041079 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1c8ba-5e1f-4c2a-8b0f-b22d8a7093dd admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:48.041392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1c8ba-5e1f-4c2a-8b0f-b22d8a7093dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:48.042118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1c8ba-5e1f-4c2a-8b0f-b22d8a7093dd admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:48.043271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1c8ba-5e1f-4c2a-8b0f-b22d8a7093dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:48.047230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1c8ba-5e1f-4c2a-8b0f-b22d8a7093dd admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:48.052138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb9da8-8ac1-4302-bacf-4bedae029d63 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:48.068718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d19eaf-ceb5-4fdc-95be-70b434d10c95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:48.068718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d19eaf-ceb5-4fdc-95be-70b434d10c95 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:48.068718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d19eaf-ceb5-4fdc-95be-70b434d10c95 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:48.069219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d19eaf-ceb5-4fdc-95be-70b434d10c95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:48.080425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d19eaf-ceb5-4fdc-95be-70b434d10c95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:48.082923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d19eaf-ceb5-4fdc-95be-70b434d10c95 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:48.103127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 925/1829] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:48.180530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 905/1830] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:47 2022] POST /identity/v3/projects => generated 385 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:48.183988 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d19eaf-ceb5-4fdc-95be-70b434d10c95 service nova] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:48.205270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d19eaf-ceb5-4fdc-95be-70b434d10c95 service nova] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:48.281749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baebf2-724c-48ec-aea4-3688f26e6d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:48.282389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baebf2-724c-48ec-aea4-3688f26e6d1f admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:48.282683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baebf2-724c-48ec-aea4-3688f26e6d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:48.282999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baebf2-724c-48ec-aea4-3688f26e6d1f admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:48.284048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baebf2-724c-48ec-aea4-3688f26e6d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:48.286247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baebf2-724c-48ec-aea4-3688f26e6d1f admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:48.461978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 926/1831] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:48 2022] POST /identity/v3/users => generated 299 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:48.468432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1c8ba-5e1f-4c2a-8b0f-b22d8a7093dd admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:48.481849 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1c8ba-5e1f-4c2a-8b0f-b22d8a7093dd admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:48.493391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffbb41-33f5-4718-b992-447a943a60a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:48.493655 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffbb41-33f5-4718-b992-447a943a60a0 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:48.494258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffbb41-33f5-4718-b992-447a943a60a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:48.494945 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffbb41-33f5-4718-b992-447a943a60a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:48.496407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffbb41-33f5-4718-b992-447a943a60a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:48.499869 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffbb41-33f5-4718-b992-447a943a60a0 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:48.520192 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 906/1832] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:05:48.526958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baebf2-724c-48ec-aea4-3688f26e6d1f admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:48.538750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baebf2-724c-48ec-aea4-3688f26e6d1f admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:48.545858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868eb895-ba98-4f7c-9531-9800e764b3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:48.550277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eb895-ba98-4f7c-9531-9800e764b3c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:48.550277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eb895-ba98-4f7c-9531-9800e764b3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:48.550277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eb895-ba98-4f7c-9531-9800e764b3c2 admin admin] PATH_INFO: `/v3/projects/5c160472ae6d4bc987441b9809a99356/users/8d7aad0bc0a34111879685609fb81ab6/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:48.567096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868eb895-ba98-4f7c-9531-9800e764b3c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c160472ae6d4bc987441b9809a99356, user_id=8d7aad0bc0a34111879685609fb81ab6, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:48.578797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868eb895-ba98-4f7c-9531-9800e764b3c2 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:48.706441 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 927/1833] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:48 2022] PUT /identity/v3/projects/5c160472ae6d4bc987441b9809a99356/users/8d7aad0bc0a34111879685609fb81ab6/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:48.712735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffbb41-33f5-4718-b992-447a943a60a0 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:48.726050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffbb41-33f5-4718-b992-447a943a60a0 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:48.825542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa9a9e-5925-4633-afbe-41bcf366688c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:48.826187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa9a9e-5925-4633-afbe-41bcf366688c admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:48.826469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa9a9e-5925-4633-afbe-41bcf366688c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:48.826747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa9a9e-5925-4633-afbe-41bcf366688c admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:48.827445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa9a9e-5925-4633-afbe-41bcf366688c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:48.829299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa9a9e-5925-4633-afbe-41bcf366688c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:48.841197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 907/1834] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:48.843237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:48.848302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868eb895-ba98-4f7c-9531-9800e764b3c2 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:48.859987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868eb895-ba98-4f7c-9531-9800e764b3c2 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:48.864858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0b6bb-bf49-4390-836c-ad348bed9893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:48.865533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0b6bb-bf49-4390-836c-ad348bed9893 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:48.865810 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0b6bb-bf49-4390-836c-ad348bed9893 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:48.866082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0b6bb-bf49-4390-836c-ad348bed9893 admin admin] PATH_INFO: `/v3/projects/5c160472ae6d4bc987441b9809a99356/users/8d7aad0bc0a34111879685609fb81ab6/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:48.868974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0b6bb-bf49-4390-836c-ad348bed9893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c160472ae6d4bc987441b9809a99356, user_id=8d7aad0bc0a34111879685609fb81ab6, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:48.871673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0b6bb-bf49-4390-836c-ad348bed9893 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:48.895646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 928/1835] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:48 2022] PUT /identity/v3/projects/5c160472ae6d4bc987441b9809a99356/users/8d7aad0bc0a34111879685609fb81ab6/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:05:48.897668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4aa68-ae38-4e38-b6c7-0b80cb86b002 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:48.898079 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4aa68-ae38-4e38-b6c7-0b80cb86b002 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:48.898326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4aa68-ae38-4e38-b6c7-0b80cb86b002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:48.948297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-9fa4aa68-ae38-4e38-b6c7-0b80cb86b002 None None] MFA Rules not processed for user `8d7aad0bc0a34111879685609fb81ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:48.950514 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9fa4aa68-ae38-4e38-b6c7-0b80cb86b002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:49.049318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 908/1836] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:49.061971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0b6bb-bf49-4390-836c-ad348bed9893 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:49.148312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0b6bb-bf49-4390-836c-ad348bed9893 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:49.152323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7837d9-15ad-448f-88ff-72dd91b19ef9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:49.152955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7837d9-15ad-448f-88ff-72dd91b19ef9 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:49.153241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7837d9-15ad-448f-88ff-72dd91b19ef9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:49.153511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7837d9-15ad-448f-88ff-72dd91b19ef9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:49.167719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7837d9-15ad-448f-88ff-72dd91b19ef9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:49.168055 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7837d9-15ad-448f-88ff-72dd91b19ef9 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:49.185810 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 929/1837] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:49.378471 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:50.858217 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4aa68-ae38-4e38-b6c7-0b80cb86b002 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:50.893197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:50.993197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4aa68-ae38-4e38-b6c7-0b80cb86b002 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:51.003132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085bbef-4516-4d97-97df-bf730a91aecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:51.003132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085bbef-4516-4d97-97df-bf730a91aecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:51.003132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085bbef-4516-4d97-97df-bf730a91aecc admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:51.003132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085bbef-4516-4d97-97df-bf730a91aecc admin admin] PATH_INFO: `/v3/users/be1bf818dd6d4b62a7ecf70ac3fec71d` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:51.006340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085bbef-4516-4d97-97df-bf730a91aecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=be1bf818dd6d4b62a7ecf70ac3fec71d)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:51.010617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085bbef-4516-4d97-97df-bf730a91aecc admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:51.172234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:51.266400 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4085bbef-4516-4d97-97df-bf730a91aecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be1bf818dd6d4b62a7ecf70ac3fec71d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:51.267595 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4085bbef-4516-4d97-97df-bf730a91aecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be1bf818dd6d4b62a7ecf70ac3fec71d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:51.269196 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4085bbef-4516-4d97-97df-bf730a91aecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be1bf818dd6d4b62a7ecf70ac3fec71d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:51.282769 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4085bbef-4516-4d97-97df-bf730a91aecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be1bf818dd6d4b62a7ecf70ac3fec71d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:51.303742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4085bbef-4516-4d97-97df-bf730a91aecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be1bf818dd6d4b62a7ecf70ac3fec71d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:51.413033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 909/1838] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:05:50 2022] DELETE /identity/v3/users/be1bf818dd6d4b62a7ecf70ac3fec71d => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:51.414836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f356487-ba38-4ed6-9b77-703f7425134a None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:51.415352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f356487-ba38-4ed6-9b77-703f7425134a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:51.416168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f356487-ba38-4ed6-9b77-703f7425134a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:51.447008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-6f356487-ba38-4ed6-9b77-703f7425134a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:51.448306 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6f356487-ba38-4ed6-9b77-703f7425134a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:51.534133 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 930/1839] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:51.549594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085bbef-4516-4d97-97df-bf730a91aecc admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:51.551092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4085bbef-4516-4d97-97df-bf730a91aecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:51.687415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085bbef-4516-4d97-97df-bf730a91aecc admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:51.692056 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3eb63-0025-42cc-9017-6c481a24ff7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:51.692735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3eb63-0025-42cc-9017-6c481a24ff7b service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:51.693143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3eb63-0025-42cc-9017-6c481a24ff7b service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:51.694301 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3eb63-0025-42cc-9017-6c481a24ff7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:51.706222 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3eb63-0025-42cc-9017-6c481a24ff7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:51.707824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3eb63-0025-42cc-9017-6c481a24ff7b service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:51.719389 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 910/1840] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:05:51.951561 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:52.142325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:52.438073 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f356487-ba38-4ed6-9b77-703f7425134a None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:52.439761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6f356487-ba38-4ed6-9b77-703f7425134a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:52.454637 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f356487-ba38-4ed6-9b77-703f7425134a None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:52.461487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:52.462288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:52.462579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:52.462900 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] PATH_INFO: `/v3/projects/e57f3bc37fcb46699068dc3a0f880e05` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:52.464015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e57f3bc37fcb46699068dc3a0f880e05)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:52.466317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:52.593266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e57f3bc37fcb46699068dc3a0f880e05', 'request_id': 'req-6dc38730-f103-40ae-a51e-2b75940a0865'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:52.596683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e57f3bc37fcb46699068dc3a0f880e05', 'request_id': 'req-6dc38730-f103-40ae-a51e-2b75940a0865'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:52.693869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e57f3bc37fcb46699068dc3a0f880e05', 'request_id': 'req-6dc38730-f103-40ae-a51e-2b75940a0865'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:52.710790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] The token cache is being invalidate because project e57f3bc37fcb46699068dc3a0f880e05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:05:52.711385 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:52.712525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:52.714309 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 931/1841] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:05:52 2022] DELETE /identity/v3/projects/e57f3bc37fcb46699068dc3a0f880e05 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:52.720481 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3eb63-0025-42cc-9017-6c481a24ff7b service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:52.732731 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3eb63-0025-42cc-9017-6c481a24ff7b service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:52.825733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44896df5-a85f-4eae-b507-e93ba0275bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:52.830376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44896df5-a85f-4eae-b507-e93ba0275bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:52.830376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44896df5-a85f-4eae-b507-e93ba0275bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:52.830376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44896df5-a85f-4eae-b507-e93ba0275bfd admin admin] PATH_INFO: `/v3/users/8d7aad0bc0a34111879685609fb81ab6` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:52.830376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44896df5-a85f-4eae-b507-e93ba0275bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d7aad0bc0a34111879685609fb81ab6)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:52.832063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44896df5-a85f-4eae-b507-e93ba0275bfd admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:53.083735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-44896df5-a85f-4eae-b507-e93ba0275bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d7aad0bc0a34111879685609fb81ab6'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:53.084460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-44896df5-a85f-4eae-b507-e93ba0275bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d7aad0bc0a34111879685609fb81ab6'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:53.085239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-44896df5-a85f-4eae-b507-e93ba0275bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d7aad0bc0a34111879685609fb81ab6'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:53.093179 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-44896df5-a85f-4eae-b507-e93ba0275bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d7aad0bc0a34111879685609fb81ab6'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:53.106565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-44896df5-a85f-4eae-b507-e93ba0275bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d7aad0bc0a34111879685609fb81ab6'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:53.203405 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:53.209530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 911/1842] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:05:52 2022] DELETE /identity/v3/users/8d7aad0bc0a34111879685609fb81ab6 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:53.853040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:53.855889 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:53.870456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc38730-f103-40ae-a51e-2b75940a0865 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:53.942756 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e5216-9ee8-4528-ae18-379003849d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:53.943433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e5216-9ee8-4528-ae18-379003849d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:53.943815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e5216-9ee8-4528-ae18-379003849d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:53.944017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e5216-9ee8-4528-ae18-379003849d9b admin admin] PATH_INFO: `/v3/projects/5c160472ae6d4bc987441b9809a99356` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:53.945224 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e5216-9ee8-4528-ae18-379003849d9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c160472ae6d4bc987441b9809a99356)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:53.947717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e5216-9ee8-4528-ae18-379003849d9b admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:54.084951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:54.160765 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6c1e5216-9ee8-4528-ae18-379003849d9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c160472ae6d4bc987441b9809a99356', 'request_id': 'req-6c1e5216-9ee8-4528-ae18-379003849d9b'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:54.170547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6c1e5216-9ee8-4528-ae18-379003849d9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c160472ae6d4bc987441b9809a99356', 'request_id': 'req-6c1e5216-9ee8-4528-ae18-379003849d9b'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:54.358451 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6c1e5216-9ee8-4528-ae18-379003849d9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c160472ae6d4bc987441b9809a99356', 'request_id': 'req-6c1e5216-9ee8-4528-ae18-379003849d9b'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:54.375828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6c1e5216-9ee8-4528-ae18-379003849d9b admin admin] The token cache is being invalidate because project 5c160472ae6d4bc987441b9809a99356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:05:54.376105 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6c1e5216-9ee8-4528-ae18-379003849d9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:54.376938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6c1e5216-9ee8-4528-ae18-379003849d9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:54.378553 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 932/1843] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:05:53 2022] DELETE /identity/v3/projects/5c160472ae6d4bc987441b9809a99356 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:54.383885 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd025de-6085-47f4-a908-fd04b20b8abd None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:54.384462 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd025de-6085-47f4-a908-fd04b20b8abd None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:54.384462 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd025de-6085-47f4-a908-fd04b20b8abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:54.405294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-dfd025de-6085-47f4-a908-fd04b20b8abd None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:54.406496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-dfd025de-6085-47f4-a908-fd04b20b8abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:54.475574 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:54.477807 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 912/1844] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:05:54.485736 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836bdc9-1de1-42d9-8cc8-0268dee75f4f None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:54.486941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836bdc9-1de1-42d9-8cc8-0268dee75f4f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:54.486941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836bdc9-1de1-42d9-8cc8-0268dee75f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:54.515065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7836bdc9-1de1-42d9-8cc8-0268dee75f4f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:54.516774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7836bdc9-1de1-42d9-8cc8-0268dee75f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:54.525916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 933/1845] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:54.530196 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd025de-6085-47f4-a908-fd04b20b8abd None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:54.541002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd025de-6085-47f4-a908-fd04b20b8abd None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:54.546420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7ff21-8ca3-4b55-a5ef-ab158f36bdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:54.547061 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7ff21-8ca3-4b55-a5ef-ab158f36bdfc admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:54.547335 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7ff21-8ca3-4b55-a5ef-ab158f36bdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:54.547646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7ff21-8ca3-4b55-a5ef-ab158f36bdfc admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:54.548421 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7ff21-8ca3-4b55-a5ef-ab158f36bdfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:05:54.548731 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7ff21-8ca3-4b55-a5ef-ab158f36bdfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:54.550333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7ff21-8ca3-4b55-a5ef-ab158f36bdfc admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:54.563802 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 913/1846] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:05:54.569290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836bdc9-1de1-42d9-8cc8-0268dee75f4f None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:54.581262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836bdc9-1de1-42d9-8cc8-0268dee75f4f None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:54.586961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a45baa-4258-4bb8-9668-fa9976e63920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:54.587629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a45baa-4258-4bb8-9668-fa9976e63920 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:54.587949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a45baa-4258-4bb8-9668-fa9976e63920 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:54.588224 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a45baa-4258-4bb8-9668-fa9976e63920 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:54.588904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a45baa-4258-4bb8-9668-fa9976e63920 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:54.591440 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a45baa-4258-4bb8-9668-fa9976e63920 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:54.710259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 934/1847] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:54 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:54.711738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7ff21-8ca3-4b55-a5ef-ab158f36bdfc admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:54.722038 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7ff21-8ca3-4b55-a5ef-ab158f36bdfc admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:54.787076 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4765f-bc65-44d1-a142-6931c19050e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:54.787866 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4765f-bc65-44d1-a142-6931c19050e5 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:54.788530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4765f-bc65-44d1-a142-6931c19050e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:54.788937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4765f-bc65-44d1-a142-6931c19050e5 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:54.789675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4765f-bc65-44d1-a142-6931c19050e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:54.791640 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4765f-bc65-44d1-a142-6931c19050e5 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:54.900819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 914/1848] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:54 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:54.905481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a45baa-4258-4bb8-9668-fa9976e63920 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:54.915930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a45baa-4258-4bb8-9668-fa9976e63920 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:54.921150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa4825-1a07-47ac-9455-8d6209bff115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:54.922011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa4825-1a07-47ac-9455-8d6209bff115 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:54.922335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa4825-1a07-47ac-9455-8d6209bff115 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:54.922672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa4825-1a07-47ac-9455-8d6209bff115 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:54.925370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa4825-1a07-47ac-9455-8d6209bff115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:54.925370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa4825-1a07-47ac-9455-8d6209bff115 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:54.936544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 935/1849] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:54.940804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4765f-bc65-44d1-a142-6931c19050e5 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:54.951920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4765f-bc65-44d1-a142-6931c19050e5 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:54.959998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd81c2-12c5-4b88-aeb3-120a0e99e6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:54.959998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd81c2-12c5-4b88-aeb3-120a0e99e6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:54.959998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd81c2-12c5-4b88-aeb3-120a0e99e6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:54.959998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd81c2-12c5-4b88-aeb3-120a0e99e6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:54.960349 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd81c2-12c5-4b88-aeb3-120a0e99e6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:54.960349 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd81c2-12c5-4b88-aeb3-120a0e99e6b1 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:54.971366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 915/1850] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:54.975993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa4825-1a07-47ac-9455-8d6209bff115 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:54.986157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:54.989847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa4825-1a07-47ac-9455-8d6209bff115 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:54.995679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f19121-fb42-4351-9378-7198f987942b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:54.996348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19121-fb42-4351-9378-7198f987942b admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:54.996628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19121-fb42-4351-9378-7198f987942b admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:54.996902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19121-fb42-4351-9378-7198f987942b admin admin] PATH_INFO: `/v3/projects/1f91d9d2594646e3a8ac053bedf273ed/users/e16d833c03114b95b37e9434f5d3659c/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:55.012830 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f19121-fb42-4351-9378-7198f987942b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f91d9d2594646e3a8ac053bedf273ed, user_id=e16d833c03114b95b37e9434f5d3659c, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:55.015494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f19121-fb42-4351-9378-7198f987942b admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:55.126332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 936/1851] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:54 2022] PUT /identity/v3/projects/1f91d9d2594646e3a8ac053bedf273ed/users/e16d833c03114b95b37e9434f5d3659c/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:55.138467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716343d-b122-4374-9630-99be34f8e372 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:55.138775 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716343d-b122-4374-9630-99be34f8e372 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:55.139011 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716343d-b122-4374-9630-99be34f8e372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:55.169045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-5716343d-b122-4374-9630-99be34f8e372 None None] MFA Rules not processed for user `e16d833c03114b95b37e9434f5d3659c`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:55.170440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-5716343d-b122-4374-9630-99be34f8e372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:55.221820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:55.253456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 916/1852] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:55.262554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f19121-fb42-4351-9378-7198f987942b admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:55.264402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-76f19121-fb42-4351-9378-7198f987942b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:55.333480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f19121-fb42-4351-9378-7198f987942b admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:55.338113 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7827d2-b091-40bb-b841-1d8b18078396 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:55.338751 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7827d2-b091-40bb-b841-1d8b18078396 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:55.339032 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7827d2-b091-40bb-b841-1d8b18078396 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:55.339348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7827d2-b091-40bb-b841-1d8b18078396 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:55.348697 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7827d2-b091-40bb-b841-1d8b18078396 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:55.350303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7827d2-b091-40bb-b841-1d8b18078396 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:55.364331 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 937/1853] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:55.385577 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716343d-b122-4374-9630-99be34f8e372 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:55.459121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716343d-b122-4374-9630-99be34f8e372 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:55.464938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b11173-8260-4697-84a3-fa00ee177cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:55.465602 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b11173-8260-4697-84a3-fa00ee177cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:55.465889 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b11173-8260-4697-84a3-fa00ee177cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:55.466172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b11173-8260-4697-84a3-fa00ee177cf3 admin admin] PATH_INFO: `/v3/users/e16d833c03114b95b37e9434f5d3659c` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:55.467425 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b11173-8260-4697-84a3-fa00ee177cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e16d833c03114b95b37e9434f5d3659c)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:55.469436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b11173-8260-4697-84a3-fa00ee177cf3 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:55.734340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:55.813544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-78b11173-8260-4697-84a3-fa00ee177cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e16d833c03114b95b37e9434f5d3659c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:55.814318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-78b11173-8260-4697-84a3-fa00ee177cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e16d833c03114b95b37e9434f5d3659c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:55.815023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-78b11173-8260-4697-84a3-fa00ee177cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e16d833c03114b95b37e9434f5d3659c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:55.825731 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-78b11173-8260-4697-84a3-fa00ee177cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e16d833c03114b95b37e9434f5d3659c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:55.851344 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-78b11173-8260-4697-84a3-fa00ee177cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e16d833c03114b95b37e9434f5d3659c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:56.049801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 917/1854] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:05:55 2022] DELETE /identity/v3/users/e16d833c03114b95b37e9434f5d3659c => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:56.055339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee549ffb-1636-4a7f-b358-4d0fa3d2da61 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:56.056103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee549ffb-1636-4a7f-b358-4d0fa3d2da61 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:56.056103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee549ffb-1636-4a7f-b358-4d0fa3d2da61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:56.076102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-ee549ffb-1636-4a7f-b358-4d0fa3d2da61 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:56.077431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ee549ffb-1636-4a7f-b358-4d0fa3d2da61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:56.151920 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 938/1855] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:56.161650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b11173-8260-4697-84a3-fa00ee177cf3 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:56.163183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-78b11173-8260-4697-84a3-fa00ee177cf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:56.235355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b11173-8260-4697-84a3-fa00ee177cf3 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:56.239067 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed56dc-d62e-402a-9ac0-0e24f9ba417e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:56.239735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed56dc-d62e-402a-9ac0-0e24f9ba417e service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:56.249018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed56dc-d62e-402a-9ac0-0e24f9ba417e service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:56.249018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed56dc-d62e-402a-9ac0-0e24f9ba417e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:56.253559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed56dc-d62e-402a-9ac0-0e24f9ba417e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:56.254992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed56dc-d62e-402a-9ac0-0e24f9ba417e service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:56.265721 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 918/1856] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:05:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:05:56.852720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee549ffb-1636-4a7f-b358-4d0fa3d2da61 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:56.852720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ee549ffb-1636-4a7f-b358-4d0fa3d2da61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:56.880621 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee549ffb-1636-4a7f-b358-4d0fa3d2da61 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:56.893456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d31f8-db73-4f39-8483-a8fd30e31862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:56.894745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d31f8-db73-4f39-8483-a8fd30e31862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:56.901516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d31f8-db73-4f39-8483-a8fd30e31862 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:56.901516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d31f8-db73-4f39-8483-a8fd30e31862 admin admin] PATH_INFO: `/v3/projects/1f91d9d2594646e3a8ac053bedf273ed` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:56.901516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d31f8-db73-4f39-8483-a8fd30e31862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f91d9d2594646e3a8ac053bedf273ed)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:56.903401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d31f8-db73-4f39-8483-a8fd30e31862 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:57.052380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:57.075350 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c09d31f8-db73-4f39-8483-a8fd30e31862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f91d9d2594646e3a8ac053bedf273ed', 'request_id': 'req-c09d31f8-db73-4f39-8483-a8fd30e31862'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:57.082652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c09d31f8-db73-4f39-8483-a8fd30e31862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f91d9d2594646e3a8ac053bedf273ed', 'request_id': 'req-c09d31f8-db73-4f39-8483-a8fd30e31862'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:57.204139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c09d31f8-db73-4f39-8483-a8fd30e31862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f91d9d2594646e3a8ac053bedf273ed', 'request_id': 'req-c09d31f8-db73-4f39-8483-a8fd30e31862'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:57.219154 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:57.224091 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c09d31f8-db73-4f39-8483-a8fd30e31862 admin admin] The token cache is being invalidate because project 1f91d9d2594646e3a8ac053bedf273ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:05:57.224377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c09d31f8-db73-4f39-8483-a8fd30e31862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:57.225079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c09d31f8-db73-4f39-8483-a8fd30e31862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:05:57.226579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 939/1857] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:05:56 2022] DELETE /identity/v3/projects/1f91d9d2594646e3a8ac053bedf273ed => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:57.234199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b2eaa-7a86-4d40-bd45-a103d3671b1f None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:57.234522 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b2eaa-7a86-4d40-bd45-a103d3671b1f None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:57.234958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b2eaa-7a86-4d40-bd45-a103d3671b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:57.258690 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-4e5b2eaa-7a86-4d40-bd45-a103d3671b1f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:57.260129 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4e5b2eaa-7a86-4d40-bd45-a103d3671b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:57.331667 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 919/1858] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:57.341342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd49bc7-d627-4c76-9c62-961a2e10835c None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:57.341843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd49bc7-d627-4c76-9c62-961a2e10835c None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:57.342189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd49bc7-d627-4c76-9c62-961a2e10835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:57.363338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7bd49bc7-d627-4c76-9c62-961a2e10835c None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:57.364598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7bd49bc7-d627-4c76-9c62-961a2e10835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:57.373185 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 940/1859] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:05:57.377704 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b2eaa-7a86-4d40-bd45-a103d3671b1f None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:57.389229 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b2eaa-7a86-4d40-bd45-a103d3671b1f None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:57.394041 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186823c-91a7-422f-b1aa-540cefad3167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:57.394671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186823c-91a7-422f-b1aa-540cefad3167 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:57.394988 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186823c-91a7-422f-b1aa-540cefad3167 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:57.395388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186823c-91a7-422f-b1aa-540cefad3167 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:57.396003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186823c-91a7-422f-b1aa-540cefad3167 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:05:57.396308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186823c-91a7-422f-b1aa-540cefad3167 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:57.397888 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186823c-91a7-422f-b1aa-540cefad3167 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:57.411955 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 920/1860] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:05:57.416521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd49bc7-d627-4c76-9c62-961a2e10835c None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:57.426033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd49bc7-d627-4c76-9c62-961a2e10835c None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:57.430625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffdffb-1236-4d68-9cd5-31329a431452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:57.431235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffdffb-1236-4d68-9cd5-31329a431452 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:57.431542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffdffb-1236-4d68-9cd5-31329a431452 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:57.431814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffdffb-1236-4d68-9cd5-31329a431452 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:57.432471 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffdffb-1236-4d68-9cd5-31329a431452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:57.434339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffdffb-1236-4d68-9cd5-31329a431452 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:57.579892 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 941/1861] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:57 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:57.584711 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186823c-91a7-422f-b1aa-540cefad3167 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:57.594030 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186823c-91a7-422f-b1aa-540cefad3167 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:57.659692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885775e-dabe-463d-b2f1-f046b45e158d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:57.660438 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885775e-dabe-463d-b2f1-f046b45e158d admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:57.660722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885775e-dabe-463d-b2f1-f046b45e158d admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:57.661010 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885775e-dabe-463d-b2f1-f046b45e158d admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:57.661713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885775e-dabe-463d-b2f1-f046b45e158d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:57.663820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885775e-dabe-463d-b2f1-f046b45e158d admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:57.814407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 921/1862] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:57 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:57.818915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffdffb-1236-4d68-9cd5-31329a431452 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:57.835180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffdffb-1236-4d68-9cd5-31329a431452 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:57.840335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9feb996-ccb7-411f-9c00-981525889980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:57.841108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9feb996-ccb7-411f-9c00-981525889980 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:57.841391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9feb996-ccb7-411f-9c00-981525889980 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:57.841662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9feb996-ccb7-411f-9c00-981525889980 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:57.842520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9feb996-ccb7-411f-9c00-981525889980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:57.844187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9feb996-ccb7-411f-9c00-981525889980 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:57.855782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 942/1863] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:05:57.861677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885775e-dabe-463d-b2f1-f046b45e158d admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:57.874453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885775e-dabe-463d-b2f1-f046b45e158d admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:57.879733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2f159-4cc7-42b1-81d1-79bf06c4f861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:57.880530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2f159-4cc7-42b1-81d1-79bf06c4f861 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:57.880912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2f159-4cc7-42b1-81d1-79bf06c4f861 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:57.881200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2f159-4cc7-42b1-81d1-79bf06c4f861 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:57.881910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2f159-4cc7-42b1-81d1-79bf06c4f861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:57.884326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2f159-4cc7-42b1-81d1-79bf06c4f861 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:57.896403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 922/1864] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:57.900963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9feb996-ccb7-411f-9c00-981525889980 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:57.911969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9feb996-ccb7-411f-9c00-981525889980 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:57.917365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a9ea9-ab24-4e66-9e78-67dcad5e8699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:57.918046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a9ea9-ab24-4e66-9e78-67dcad5e8699 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:57.918361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a9ea9-ab24-4e66-9e78-67dcad5e8699 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:57.918634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a9ea9-ab24-4e66-9e78-67dcad5e8699 admin admin] PATH_INFO: `/v3/projects/34130fec4ec3442285c9c4327531b33e/users/db3c75ad89e54835934610d253194ca2/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:57.935423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a9ea9-ab24-4e66-9e78-67dcad5e8699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34130fec4ec3442285c9c4327531b33e, user_id=db3c75ad89e54835934610d253194ca2, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:57.938250 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a9ea9-ab24-4e66-9e78-67dcad5e8699 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:58.050604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 943/1865] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:57 2022] PUT /identity/v3/projects/34130fec4ec3442285c9c4327531b33e/users/db3c75ad89e54835934610d253194ca2/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:58.064487 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e25be-b5b6-43a3-b390-3fb828bf30f9 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:58.064487 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e25be-b5b6-43a3-b390-3fb828bf30f9 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:58.064487 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e25be-b5b6-43a3-b390-3fb828bf30f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:58.103904 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-df8e25be-b5b6-43a3-b390-3fb828bf30f9 None None] MFA Rules not processed for user `db3c75ad89e54835934610d253194ca2`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:58.105509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-df8e25be-b5b6-43a3-b390-3fb828bf30f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:58.181675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 923/1866] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:58.187772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a9ea9-ab24-4e66-9e78-67dcad5e8699 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:58.266886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a9ea9-ab24-4e66-9e78-67dcad5e8699 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:58.271754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e5aed-ae8b-45dc-b346-02d918d3b577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:58.272460 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e5aed-ae8b-45dc-b346-02d918d3b577 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:58.272817 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e5aed-ae8b-45dc-b346-02d918d3b577 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:58.273163 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e5aed-ae8b-45dc-b346-02d918d3b577 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:58.274329 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e5aed-ae8b-45dc-b346-02d918d3b577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:58.276909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e5aed-ae8b-45dc-b346-02d918d3b577 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:58.405058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:58.415868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 944/1867] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:58 2022] POST /identity/v3/projects => generated 383 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:05:58.417224 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e25be-b5b6-43a3-b390-3fb828bf30f9 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:58.428769 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e25be-b5b6-43a3-b390-3fb828bf30f9 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:58.496424 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6f827-6f35-4984-a52e-654e9fffbcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:58.497053 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6f827-6f35-4984-a52e-654e9fffbcb4 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:58.497340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6f827-6f35-4984-a52e-654e9fffbcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:58.498069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6f827-6f35-4984-a52e-654e9fffbcb4 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:58.498559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6f827-6f35-4984-a52e-654e9fffbcb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:58.500486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6f827-6f35-4984-a52e-654e9fffbcb4 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:58.730910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 924/1868] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:58 2022] POST /identity/v3/users => generated 292 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:05:58.735057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e5aed-ae8b-45dc-b346-02d918d3b577 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:58.747960 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e5aed-ae8b-45dc-b346-02d918d3b577 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:58.755897 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500ebef-09ae-45b4-a261-fff725b52425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:58.756561 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500ebef-09ae-45b4-a261-fff725b52425 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:58.756838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500ebef-09ae-45b4-a261-fff725b52425 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:58.757115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500ebef-09ae-45b4-a261-fff725b52425 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:58.757946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a500ebef-09ae-45b4-a261-fff725b52425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:58.759557 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a500ebef-09ae-45b4-a261-fff725b52425 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:58.770847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 945/1869] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:05:58.775156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6f827-6f35-4984-a52e-654e9fffbcb4 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:58.785253 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6f827-6f35-4984-a52e-654e9fffbcb4 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:58.793290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83ae44-1dde-4a8b-ae06-daf4eb8c9019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:58.794117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83ae44-1dde-4a8b-ae06-daf4eb8c9019 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:58.794465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83ae44-1dde-4a8b-ae06-daf4eb8c9019 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:58.794810 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83ae44-1dde-4a8b-ae06-daf4eb8c9019 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:58.795620 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83ae44-1dde-4a8b-ae06-daf4eb8c9019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:58.797289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83ae44-1dde-4a8b-ae06-daf4eb8c9019 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:58.812363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 925/1870] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:58.815251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500ebef-09ae-45b4-a261-fff725b52425 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:58.829961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500ebef-09ae-45b4-a261-fff725b52425 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:58.841720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca485769-4071-4bff-9fb9-6376f5aa48c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:58.841720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca485769-4071-4bff-9fb9-6376f5aa48c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:58.841720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca485769-4071-4bff-9fb9-6376f5aa48c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:58.842092 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca485769-4071-4bff-9fb9-6376f5aa48c7 admin admin] PATH_INFO: `/v3/projects/6aa80fa7719448ceb753335264aca52a/users/fbf6e928e6414b9c9b7f3b5b076fb2e1/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:58.859628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca485769-4071-4bff-9fb9-6376f5aa48c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aa80fa7719448ceb753335264aca52a, user_id=fbf6e928e6414b9c9b7f3b5b076fb2e1, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:58.862670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca485769-4071-4bff-9fb9-6376f5aa48c7 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:58.981349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 946/1871] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:58 2022] PUT /identity/v3/projects/6aa80fa7719448ceb753335264aca52a/users/fbf6e928e6414b9c9b7f3b5b076fb2e1/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:58.990329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746e251-75f0-4486-b54e-8379ed1467a9 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:58.992792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746e251-75f0-4486-b54e-8379ed1467a9 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:58.992792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746e251-75f0-4486-b54e-8379ed1467a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:59.029993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-5746e251-75f0-4486-b54e-8379ed1467a9 None None] MFA Rules not processed for user `fbf6e928e6414b9c9b7f3b5b076fb2e1`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:05:59.031227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-5746e251-75f0-4486-b54e-8379ed1467a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:05:59.050861 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:59.105355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 926/1872] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:05:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:05:59.110210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca485769-4071-4bff-9fb9-6376f5aa48c7 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:59.161855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:59.180120 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca485769-4071-4bff-9fb9-6376f5aa48c7 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:59.185935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8709666-3ee0-41df-8207-79f86c573a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:59.186607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8709666-3ee0-41df-8207-79f86c573a80 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:59.187275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8709666-3ee0-41df-8207-79f86c573a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:59.187275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8709666-3ee0-41df-8207-79f86c573a80 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:59.189279 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8709666-3ee0-41df-8207-79f86c573a80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:59.190053 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8709666-3ee0-41df-8207-79f86c573a80 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:59.332398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 947/1873] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:05:59 2022] POST /identity/v3/projects => generated 383 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:05:59.337471 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746e251-75f0-4486-b54e-8379ed1467a9 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:59.348792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746e251-75f0-4486-b54e-8379ed1467a9 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:59.394176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:05:59.412334 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8181e8-1650-4e61-808f-95bcc4103045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:59.412964 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8181e8-1650-4e61-808f-95bcc4103045 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:59.413250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8181e8-1650-4e61-808f-95bcc4103045 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:59.413521 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8181e8-1650-4e61-808f-95bcc4103045 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:59.415316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8181e8-1650-4e61-808f-95bcc4103045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:59.416731 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8181e8-1650-4e61-808f-95bcc4103045 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:59.583423 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 927/1874] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:05:59 2022] POST /identity/v3/users => generated 298 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:05:59.590039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8709666-3ee0-41df-8207-79f86c573a80 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:59.615928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8709666-3ee0-41df-8207-79f86c573a80 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:59.627031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2e0c0-4223-4e8d-97e8-41ea112bf799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:59.627829 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2e0c0-4223-4e8d-97e8-41ea112bf799 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:59.628149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2e0c0-4223-4e8d-97e8-41ea112bf799 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:59.628472 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2e0c0-4223-4e8d-97e8-41ea112bf799 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:59.629870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2e0c0-4223-4e8d-97e8-41ea112bf799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:59.632081 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2e0c0-4223-4e8d-97e8-41ea112bf799 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:59.689071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8181e8-1650-4e61-808f-95bcc4103045 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:59.689547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 948/1875] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:05:59.705288 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8181e8-1650-4e61-808f-95bcc4103045 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:05:59.720469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae74c81-cec5-42d2-9bc1-c4e3f1ae344b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:05:59.721199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae74c81-cec5-42d2-9bc1-c4e3f1ae344b admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:05:59.721503 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae74c81-cec5-42d2-9bc1-c4e3f1ae344b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:05:59.721899 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae74c81-cec5-42d2-9bc1-c4e3f1ae344b admin admin] PATH_INFO: `/v3/projects/a203d4f4533049dd9dff836964badbf7/users/2066a43ee12b4c168af072b34711047f/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:05:59.744332 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae74c81-cec5-42d2-9bc1-c4e3f1ae344b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a203d4f4533049dd9dff836964badbf7, user_id=2066a43ee12b4c168af072b34711047f, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:05:59.747071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae74c81-cec5-42d2-9bc1-c4e3f1ae344b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:05:59.945914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 928/1876] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:05:59 2022] PUT /identity/v3/projects/a203d4f4533049dd9dff836964badbf7/users/2066a43ee12b4c168af072b34711047f/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:05:59.951634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2e0c0-4223-4e8d-97e8-41ea112bf799 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:05:59.963710 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2e0c0-4223-4e8d-97e8-41ea112bf799 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:00.037551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898548e-d799-4fa1-a649-c68808f9fc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:00.038201 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898548e-d799-4fa1-a649-c68808f9fc21 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:00.038480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898548e-d799-4fa1-a649-c68808f9fc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:00.038761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898548e-d799-4fa1-a649-c68808f9fc21 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:00.039922 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898548e-d799-4fa1-a649-c68808f9fc21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:00.041515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898548e-d799-4fa1-a649-c68808f9fc21 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:00.054185 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 949/1877] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:06:00.060890 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae74c81-cec5-42d2-9bc1-c4e3f1ae344b admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:00.080431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae74c81-cec5-42d2-9bc1-c4e3f1ae344b admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:00.088666 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be668c3-2851-42d6-9d59-6b1173209454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:00.089502 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be668c3-2851-42d6-9d59-6b1173209454 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:00.089872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be668c3-2851-42d6-9d59-6b1173209454 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:00.090236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be668c3-2851-42d6-9d59-6b1173209454 admin admin] PATH_INFO: `/v3/projects/a203d4f4533049dd9dff836964badbf7/users/2066a43ee12b4c168af072b34711047f/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:00.096066 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be668c3-2851-42d6-9d59-6b1173209454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a203d4f4533049dd9dff836964badbf7, user_id=2066a43ee12b4c168af072b34711047f, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:00.100266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be668c3-2851-42d6-9d59-6b1173209454 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:00.139609 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 929/1878] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:00 2022] PUT /identity/v3/projects/a203d4f4533049dd9dff836964badbf7/users/2066a43ee12b4c168af072b34711047f/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:00.160985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ca04f-256f-488e-8685-41a4a3758afa None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:00.161281 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ca04f-256f-488e-8685-41a4a3758afa None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:00.161555 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ca04f-256f-488e-8685-41a4a3758afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:00.202883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-095ca04f-256f-488e-8685-41a4a3758afa None None] MFA Rules not processed for user `2066a43ee12b4c168af072b34711047f`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:00.204210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-095ca04f-256f-488e-8685-41a4a3758afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:00.351517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 950/1879] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:00.558401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ca04f-256f-488e-8685-41a4a3758afa None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:00.559532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be668c3-2851-42d6-9d59-6b1173209454 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:00.559954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-095ca04f-256f-488e-8685-41a4a3758afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:00.562707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-6be668c3-2851-42d6-9d59-6b1173209454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:00.627923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:00.645267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ca04f-256f-488e-8685-41a4a3758afa None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:00.653671 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f30e65-9cad-42f6-962d-8904253fd3d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:00.656010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f30e65-9cad-42f6-962d-8904253fd3d7 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:00.656620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f30e65-9cad-42f6-962d-8904253fd3d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:00.657233 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f30e65-9cad-42f6-962d-8904253fd3d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:00.668838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be668c3-2851-42d6-9d59-6b1173209454 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:00.673900 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823b6cd-2bcf-43c7-a1a2-51ecd8201976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:00.682540 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823b6cd-2bcf-43c7-a1a2-51ecd8201976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:00.682540 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823b6cd-2bcf-43c7-a1a2-51ecd8201976 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:00.682540 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823b6cd-2bcf-43c7-a1a2-51ecd8201976 admin admin] PATH_INFO: `/v3/users/45765858d2cb4bbfa0ba15947b814a7d` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:00.682540 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823b6cd-2bcf-43c7-a1a2-51ecd8201976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45765858d2cb4bbfa0ba15947b814a7d)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:00.682540 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823b6cd-2bcf-43c7-a1a2-51ecd8201976 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:00.748752 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f30e65-9cad-42f6-962d-8904253fd3d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:00.750738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f30e65-9cad-42f6-962d-8904253fd3d7 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:00.761547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 951/1880] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:06:00.911816 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3823b6cd-2bcf-43c7-a1a2-51ecd8201976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45765858d2cb4bbfa0ba15947b814a7d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:00.912955 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3823b6cd-2bcf-43c7-a1a2-51ecd8201976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45765858d2cb4bbfa0ba15947b814a7d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:00.913251 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3823b6cd-2bcf-43c7-a1a2-51ecd8201976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45765858d2cb4bbfa0ba15947b814a7d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:00.924476 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3823b6cd-2bcf-43c7-a1a2-51ecd8201976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45765858d2cb4bbfa0ba15947b814a7d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:00.944792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3823b6cd-2bcf-43c7-a1a2-51ecd8201976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45765858d2cb4bbfa0ba15947b814a7d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:01.044206 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 930/1881] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:00 2022] DELETE /identity/v3/users/45765858d2cb4bbfa0ba15947b814a7d => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:01.048786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdb54e-86a7-4bdc-b3ab-225f6125f10a None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:01.049150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdb54e-86a7-4bdc-b3ab-225f6125f10a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:01.049546 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdb54e-86a7-4bdc-b3ab-225f6125f10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:01.074149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-2abdb54e-86a7-4bdc-b3ab-225f6125f10a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:01.074149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2abdb54e-86a7-4bdc-b3ab-225f6125f10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:01.143287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 952/1882] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:01.151635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823b6cd-2bcf-43c7-a1a2-51ecd8201976 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:01.153113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3823b6cd-2bcf-43c7-a1a2-51ecd8201976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:01.186792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:01.238298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823b6cd-2bcf-43c7-a1a2-51ecd8201976 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:01.242849 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c93da-0e51-46ae-8d33-80eb324115a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:01.259569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c93da-0e51-46ae-8d33-80eb324115a6 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:01.259569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c93da-0e51-46ae-8d33-80eb324115a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:01.259569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c93da-0e51-46ae-8d33-80eb324115a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:01.259569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c93da-0e51-46ae-8d33-80eb324115a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:01.259569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c93da-0e51-46ae-8d33-80eb324115a6 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:01.270430 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 931/1883] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:01.860333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdb54e-86a7-4bdc-b3ab-225f6125f10a None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:01.862092 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2abdb54e-86a7-4bdc-b3ab-225f6125f10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:01.881236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdb54e-86a7-4bdc-b3ab-225f6125f10a None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:01.899605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:01.900268 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:01.900552 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:01.900828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] PATH_INFO: `/v3/projects/1bfd7adf45434acc8473bd7b7b1a5fcb` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:01.902024 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bfd7adf45434acc8473bd7b7b1a5fcb)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:01.904032 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:02.067487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bfd7adf45434acc8473bd7b7b1a5fcb', 'request_id': 'req-610b132e-ebc2-4b6a-bf33-7bf5b3413362'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:02.074374 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bfd7adf45434acc8473bd7b7b1a5fcb', 'request_id': 'req-610b132e-ebc2-4b6a-bf33-7bf5b3413362'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:02.153138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bfd7adf45434acc8473bd7b7b1a5fcb', 'request_id': 'req-610b132e-ebc2-4b6a-bf33-7bf5b3413362'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:02.165181 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] The token cache is being invalidate because project 1bfd7adf45434acc8473bd7b7b1a5fcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:02.165554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:02.166275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:02.168122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 953/1884] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:01 2022] DELETE /identity/v3/projects/1bfd7adf45434acc8473bd7b7b1a5fcb => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:02.173288 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c93da-0e51-46ae-8d33-80eb324115a6 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:02.184388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c93da-0e51-46ae-8d33-80eb324115a6 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:02.255815 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a8644-6397-4f01-995b-d15c8ba3e716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:02.256474 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a8644-6397-4f01-995b-d15c8ba3e716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:02.256760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a8644-6397-4f01-995b-d15c8ba3e716 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:02.257038 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a8644-6397-4f01-995b-d15c8ba3e716 admin admin] PATH_INFO: `/v3/users/0228360f03a04e0fb4201f58d01e47f4` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:02.259261 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a8644-6397-4f01-995b-d15c8ba3e716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0228360f03a04e0fb4201f58d01e47f4)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:02.261204 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a8644-6397-4f01-995b-d15c8ba3e716 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:02.567007 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-672a8644-6397-4f01-995b-d15c8ba3e716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0228360f03a04e0fb4201f58d01e47f4'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:02.567761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-672a8644-6397-4f01-995b-d15c8ba3e716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0228360f03a04e0fb4201f58d01e47f4'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:02.568372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-672a8644-6397-4f01-995b-d15c8ba3e716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0228360f03a04e0fb4201f58d01e47f4'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:02.575888 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-672a8644-6397-4f01-995b-d15c8ba3e716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0228360f03a04e0fb4201f58d01e47f4'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:02.588533 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-672a8644-6397-4f01-995b-d15c8ba3e716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0228360f03a04e0fb4201f58d01e47f4'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:02.825053 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 932/1885] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:02 2022] DELETE /identity/v3/users/0228360f03a04e0fb4201f58d01e47f4 => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:03.084231 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:03.091334 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:03.115850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:03.155390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b132e-ebc2-4b6a-bf33-7bf5b3413362 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:03.219110 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5db914-88f4-4469-8302-2d50835c40de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:03.220423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5db914-88f4-4469-8302-2d50835c40de service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:03.220708 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5db914-88f4-4469-8302-2d50835c40de service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:03.220985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5db914-88f4-4469-8302-2d50835c40de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:03.223377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6e5db914-88f4-4469-8302-2d50835c40de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:03.314833 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5db914-88f4-4469-8302-2d50835c40de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:03.324033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5db914-88f4-4469-8302-2d50835c40de service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:03.345580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 954/1886] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:03.467884 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a8644-6397-4f01-995b-d15c8ba3e716 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:03.470115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-672a8644-6397-4f01-995b-d15c8ba3e716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:03.521120 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:03.547273 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a8644-6397-4f01-995b-d15c8ba3e716 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:03.552029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9cf746-0606-4073-a239-952b61b63a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:03.552681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9cf746-0606-4073-a239-952b61b63a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:03.552994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9cf746-0606-4073-a239-952b61b63a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:03.553268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9cf746-0606-4073-a239-952b61b63a8a admin admin] PATH_INFO: `/v3/projects/e49eef18ae0b4e31838c4e66ab9fd999` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:03.555170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9cf746-0606-4073-a239-952b61b63a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e49eef18ae0b4e31838c4e66ab9fd999)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:03.557104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9cf746-0606-4073-a239-952b61b63a8a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:03.707170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-9c9cf746-0606-4073-a239-952b61b63a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e49eef18ae0b4e31838c4e66ab9fd999', 'request_id': 'req-9c9cf746-0606-4073-a239-952b61b63a8a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:03.715399 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-9c9cf746-0606-4073-a239-952b61b63a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e49eef18ae0b4e31838c4e66ab9fd999', 'request_id': 'req-9c9cf746-0606-4073-a239-952b61b63a8a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:03.724014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-9c9cf746-0606-4073-a239-952b61b63a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e49eef18ae0b4e31838c4e66ab9fd999', 'request_id': 'req-9c9cf746-0606-4073-a239-952b61b63a8a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:03.895966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-9c9cf746-0606-4073-a239-952b61b63a8a admin admin] The token cache is being invalidate because project e49eef18ae0b4e31838c4e66ab9fd999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:03.896329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-9c9cf746-0606-4073-a239-952b61b63a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:03.897314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-9c9cf746-0606-4073-a239-952b61b63a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:03.899755 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 933/1887] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:03 2022] DELETE /identity/v3/projects/e49eef18ae0b4e31838c4e66ab9fd999 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:03.905591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6351722-6eab-4676-a1ae-00f99ed0c9ef None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:03.905879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6351722-6eab-4676-a1ae-00f99ed0c9ef None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:03.906155 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6351722-6eab-4676-a1ae-00f99ed0c9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:03.928888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c6351722-6eab-4676-a1ae-00f99ed0c9ef None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:03.930124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c6351722-6eab-4676-a1ae-00f99ed0c9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:04.001355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 955/1888] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:04.010533 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cb992-4bed-4562-97d5-eef11c1fae51 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:04.010841 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cb992-4bed-4562-97d5-eef11c1fae51 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:04.011103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cb992-4bed-4562-97d5-eef11c1fae51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:04.033365 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-058cb992-4bed-4562-97d5-eef11c1fae51 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:04.034586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-058cb992-4bed-4562-97d5-eef11c1fae51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:04.043436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 934/1889] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:04.048830 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6351722-6eab-4676-a1ae-00f99ed0c9ef None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:04.059625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6351722-6eab-4676-a1ae-00f99ed0c9ef None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:04.064735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbd4b9-d4e7-4344-96bc-23fa7cd93c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:04.065370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbd4b9-d4e7-4344-96bc-23fa7cd93c89 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:04.065652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbd4b9-d4e7-4344-96bc-23fa7cd93c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:04.065926 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbd4b9-d4e7-4344-96bc-23fa7cd93c89 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:04.066639 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbd4b9-d4e7-4344-96bc-23fa7cd93c89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:04.066941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbd4b9-d4e7-4344-96bc-23fa7cd93c89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:04.068573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbd4b9-d4e7-4344-96bc-23fa7cd93c89 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:04.082705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 956/1890] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:06:04.088218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058cb992-4bed-4562-97d5-eef11c1fae51 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:04.098022 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058cb992-4bed-4562-97d5-eef11c1fae51 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:04.103366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba93eac-8a45-40f6-88d4-9030a7a79bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:04.104442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93eac-8a45-40f6-88d4-9030a7a79bff admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:04.105213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93eac-8a45-40f6-88d4-9030a7a79bff admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:04.105624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93eac-8a45-40f6-88d4-9030a7a79bff admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:04.106759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba93eac-8a45-40f6-88d4-9030a7a79bff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:04.109724 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba93eac-8a45-40f6-88d4-9030a7a79bff admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:04.299281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 935/1891] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:04 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:04.304008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbd4b9-d4e7-4344-96bc-23fa7cd93c89 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:04.315934 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbd4b9-d4e7-4344-96bc-23fa7cd93c89 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:04.332362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:04.393439 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e864b-c8ed-4751-b929-6c3c9fddecc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:04.394232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e864b-c8ed-4751-b929-6c3c9fddecc2 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:04.394595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e864b-c8ed-4751-b929-6c3c9fddecc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:04.394910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e864b-c8ed-4751-b929-6c3c9fddecc2 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:04.395719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e864b-c8ed-4751-b929-6c3c9fddecc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:04.398080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e864b-c8ed-4751-b929-6c3c9fddecc2 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:04.559002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 957/1892] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:04 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:04.560700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba93eac-8a45-40f6-88d4-9030a7a79bff admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:04.574624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba93eac-8a45-40f6-88d4-9030a7a79bff admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:04.580084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8e02f-b021-4256-adc0-9c4dec3201e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:04.580724 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8e02f-b021-4256-adc0-9c4dec3201e3 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:04.581018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8e02f-b021-4256-adc0-9c4dec3201e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:04.581294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8e02f-b021-4256-adc0-9c4dec3201e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:04.582008 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8e02f-b021-4256-adc0-9c4dec3201e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:04.583868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8e02f-b021-4256-adc0-9c4dec3201e3 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:04.596846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 936/1893] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:04.601620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e864b-c8ed-4751-b929-6c3c9fddecc2 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:04.612774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e864b-c8ed-4751-b929-6c3c9fddecc2 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:04.619714 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddf693-d6ff-49c0-8dbb-d3bd33aa53df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:04.620716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddf693-d6ff-49c0-8dbb-d3bd33aa53df admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:04.621158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddf693-d6ff-49c0-8dbb-d3bd33aa53df admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:04.621572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddf693-d6ff-49c0-8dbb-d3bd33aa53df admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:04.622658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddf693-d6ff-49c0-8dbb-d3bd33aa53df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:04.624841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddf693-d6ff-49c0-8dbb-d3bd33aa53df admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:04.640420 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 958/1894] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:04.645031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8e02f-b021-4256-adc0-9c4dec3201e3 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:04.656778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8e02f-b021-4256-adc0-9c4dec3201e3 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:04.668280 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ebb27-a743-4af3-825b-bb031779ac20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:04.674566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ebb27-a743-4af3-825b-bb031779ac20 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:04.674991 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ebb27-a743-4af3-825b-bb031779ac20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:04.679065 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ebb27-a743-4af3-825b-bb031779ac20 admin admin] PATH_INFO: `/v3/projects/6bc379478e7c42ab8e94bcccf4688a57/users/719447361cd045d58025d550456b9e5d/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:04.700418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ebb27-a743-4af3-825b-bb031779ac20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bc379478e7c42ab8e94bcccf4688a57, user_id=719447361cd045d58025d550456b9e5d, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:04.703285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ebb27-a743-4af3-825b-bb031779ac20 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:04.816274 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 937/1895] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:04 2022] PUT /identity/v3/projects/6bc379478e7c42ab8e94bcccf4688a57/users/719447361cd045d58025d550456b9e5d/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:04.837233 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab40b-31f3-4ba8-897e-bb5f2b791687 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:04.837600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab40b-31f3-4ba8-897e-bb5f2b791687 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:04.837938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ab40b-31f3-4ba8-897e-bb5f2b791687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:04.869677 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-d56ab40b-31f3-4ba8-897e-bb5f2b791687 None None] MFA Rules not processed for user `719447361cd045d58025d550456b9e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:04.871884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d56ab40b-31f3-4ba8-897e-bb5f2b791687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:04.951586 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 959/1896] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:04.956485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ebb27-a743-4af3-825b-bb031779ac20 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:05.033293 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ebb27-a743-4af3-825b-bb031779ac20 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:05.038220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90583fd1-c373-4cfa-9e94-6d635006927a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:05.039347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90583fd1-c373-4cfa-9e94-6d635006927a admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:05.039854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90583fd1-c373-4cfa-9e94-6d635006927a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:05.040340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90583fd1-c373-4cfa-9e94-6d635006927a admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:05.041048 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90583fd1-c373-4cfa-9e94-6d635006927a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:05.042962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90583fd1-c373-4cfa-9e94-6d635006927a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:05.053569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:05.189758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 938/1897] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:04 2022] POST /identity/v3/projects => generated 389 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:05.196228 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ab40b-31f3-4ba8-897e-bb5f2b791687 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:05.211119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ab40b-31f3-4ba8-897e-bb5f2b791687 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:05.394716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56529a62-1ae9-4c8f-9f28-06a360e2329c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:05.394716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56529a62-1ae9-4c8f-9f28-06a360e2329c admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:05.395718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56529a62-1ae9-4c8f-9f28-06a360e2329c admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:05.395992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56529a62-1ae9-4c8f-9f28-06a360e2329c admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:05.396681 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56529a62-1ae9-4c8f-9f28-06a360e2329c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:05.399916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56529a62-1ae9-4c8f-9f28-06a360e2329c admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:05.408735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90583fd1-c373-4cfa-9e94-6d635006927a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:05.411349 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-90583fd1-c373-4cfa-9e94-6d635006927a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:05.494268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90583fd1-c373-4cfa-9e94-6d635006927a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:05.498793 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc294ac-ca91-4cd4-9152-f17e0bc8c223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:05.499584 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc294ac-ca91-4cd4-9152-f17e0bc8c223 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:05.499875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc294ac-ca91-4cd4-9152-f17e0bc8c223 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:05.500163 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc294ac-ca91-4cd4-9152-f17e0bc8c223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:05.503513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 960/1898] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:05 2022] POST /identity/v3/users => generated 301 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:05.505301 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56529a62-1ae9-4c8f-9f28-06a360e2329c admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:05.509082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-bcc294ac-ca91-4cd4-9152-f17e0bc8c223 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:05.519753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56529a62-1ae9-4c8f-9f28-06a360e2329c admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:05.525970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59719db1-20b8-49d3-81a4-0b09b7149d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:05.526760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59719db1-20b8-49d3-81a4-0b09b7149d2b admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:05.527146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59719db1-20b8-49d3-81a4-0b09b7149d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:05.527549 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59719db1-20b8-49d3-81a4-0b09b7149d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:05.528345 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59719db1-20b8-49d3-81a4-0b09b7149d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:05.530071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59719db1-20b8-49d3-81a4-0b09b7149d2b admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:05.545096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 961/1899] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:05.548913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59719db1-20b8-49d3-81a4-0b09b7149d2b admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:05.555884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:05.560029 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59719db1-20b8-49d3-81a4-0b09b7149d2b admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:05.566232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170636bc-8c0b-42fc-837c-0ce61cfaead5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:05.567098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170636bc-8c0b-42fc-837c-0ce61cfaead5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:05.567381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170636bc-8c0b-42fc-837c-0ce61cfaead5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:05.567686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170636bc-8c0b-42fc-837c-0ce61cfaead5 admin admin] PATH_INFO: `/v3/projects/463b3f351a814862987777cbf5690160/users/6807711bf52a4f2ab5129e3c84ec437b/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:05.586444 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170636bc-8c0b-42fc-837c-0ce61cfaead5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=463b3f351a814862987777cbf5690160, user_id=6807711bf52a4f2ab5129e3c84ec437b, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:05.589396 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170636bc-8c0b-42fc-837c-0ce61cfaead5 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:05.624935 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc294ac-ca91-4cd4-9152-f17e0bc8c223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:05.626413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc294ac-ca91-4cd4-9152-f17e0bc8c223 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:05.638766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 939/1900] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:06:05.666083 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 962/1901] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:05 2022] PUT /identity/v3/projects/463b3f351a814862987777cbf5690160/users/6807711bf52a4f2ab5129e3c84ec437b/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:05.671195 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc294ac-ca91-4cd4-9152-f17e0bc8c223 service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:05.686543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc294ac-ca91-4cd4-9152-f17e0bc8c223 service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:05.809975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814af9ac-b12e-44f4-a0c2-505aa4fc0960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:05.811046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814af9ac-b12e-44f4-a0c2-505aa4fc0960 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:05.811582 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814af9ac-b12e-44f4-a0c2-505aa4fc0960 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:05.812083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814af9ac-b12e-44f4-a0c2-505aa4fc0960 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:05.813435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814af9ac-b12e-44f4-a0c2-505aa4fc0960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:05.816834 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814af9ac-b12e-44f4-a0c2-505aa4fc0960 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:05.836700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 940/1902] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:06:05.842931 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170636bc-8c0b-42fc-837c-0ce61cfaead5 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:05.862272 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170636bc-8c0b-42fc-837c-0ce61cfaead5 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:05.870322 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ace5ee-bcd3-4c09-9391-46faeed9f0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:05.871554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ace5ee-bcd3-4c09-9391-46faeed9f0ea admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:05.872063 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ace5ee-bcd3-4c09-9391-46faeed9f0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:05.872855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ace5ee-bcd3-4c09-9391-46faeed9f0ea admin admin] PATH_INFO: `/v3/projects/463b3f351a814862987777cbf5690160/users/6807711bf52a4f2ab5129e3c84ec437b/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:05.876545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ace5ee-bcd3-4c09-9391-46faeed9f0ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=463b3f351a814862987777cbf5690160, user_id=6807711bf52a4f2ab5129e3c84ec437b, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:05.880203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ace5ee-bcd3-4c09-9391-46faeed9f0ea admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:05.907887 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 963/1903] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:05 2022] PUT /identity/v3/projects/463b3f351a814862987777cbf5690160/users/6807711bf52a4f2ab5129e3c84ec437b/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:05.915298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420c7cb-e21d-4dcb-9aa0-c74bde7373ff None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:05.916017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420c7cb-e21d-4dcb-9aa0-c74bde7373ff None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:05.916557 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420c7cb-e21d-4dcb-9aa0-c74bde7373ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:05.962848 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-e420c7cb-e21d-4dcb-9aa0-c74bde7373ff None None] MFA Rules not processed for user `6807711bf52a4f2ab5129e3c84ec437b`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:05.964403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e420c7cb-e21d-4dcb-9aa0-c74bde7373ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:05.996218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:06.057781 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 941/1904] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:06.071326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ace5ee-bcd3-4c09-9391-46faeed9f0ea admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:06.168624 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ace5ee-bcd3-4c09-9391-46faeed9f0ea admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:06.172744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1c446-c146-473b-88bb-d9d7ccf58c28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:06.173421 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1c446-c146-473b-88bb-d9d7ccf58c28 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:06.173859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1c446-c146-473b-88bb-d9d7ccf58c28 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:06.174242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1c446-c146-473b-88bb-d9d7ccf58c28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:06.186154 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1c446-c146-473b-88bb-d9d7ccf58c28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:06.187656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1c446-c146-473b-88bb-d9d7ccf58c28 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:06.198780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 964/1905] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:06.355349 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420c7cb-e21d-4dcb-9aa0-c74bde7373ff None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:06.428942 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420c7cb-e21d-4dcb-9aa0-c74bde7373ff None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:06.434690 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3ef8d-d157-44c4-8acf-b5dd72d131f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:06.435425 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3ef8d-d157-44c4-8acf-b5dd72d131f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:06.435733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3ef8d-d157-44c4-8acf-b5dd72d131f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:06.436034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3ef8d-d157-44c4-8acf-b5dd72d131f6 admin admin] PATH_INFO: `/v3/users/719447361cd045d58025d550456b9e5d` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:06.437608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3ef8d-d157-44c4-8acf-b5dd72d131f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=719447361cd045d58025d550456b9e5d)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:06.439648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3ef8d-d157-44c4-8acf-b5dd72d131f6 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:06.735493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-02d3ef8d-d157-44c4-8acf-b5dd72d131f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '719447361cd045d58025d550456b9e5d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:06.736544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-02d3ef8d-d157-44c4-8acf-b5dd72d131f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '719447361cd045d58025d550456b9e5d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:06.737258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-02d3ef8d-d157-44c4-8acf-b5dd72d131f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '719447361cd045d58025d550456b9e5d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:06.746657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-02d3ef8d-d157-44c4-8acf-b5dd72d131f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '719447361cd045d58025d550456b9e5d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:06.767659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-02d3ef8d-d157-44c4-8acf-b5dd72d131f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '719447361cd045d58025d550456b9e5d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:06.857422 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 942/1906] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:06 2022] DELETE /identity/v3/users/719447361cd045d58025d550456b9e5d => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:06.864141 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62155cab-4825-4ae7-9765-c391318d265a None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:06.865003 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62155cab-4825-4ae7-9765-c391318d265a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:06.865429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62155cab-4825-4ae7-9765-c391318d265a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:06.888609 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-62155cab-4825-4ae7-9765-c391318d265a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:06.890086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-62155cab-4825-4ae7-9765-c391318d265a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:06.970675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 965/1907] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:06.982162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3ef8d-d157-44c4-8acf-b5dd72d131f6 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:06.985178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-02d3ef8d-d157-44c4-8acf-b5dd72d131f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:07.056800 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:07.075588 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3ef8d-d157-44c4-8acf-b5dd72d131f6 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:07.079441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26871f-603e-4ddc-b345-5610cb4ecce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:07.080099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26871f-603e-4ddc-b345-5610cb4ecce1 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:07.080382 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26871f-603e-4ddc-b345-5610cb4ecce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:07.080763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26871f-603e-4ddc-b345-5610cb4ecce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:07.092819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26871f-603e-4ddc-b345-5610cb4ecce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:07.094187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26871f-603e-4ddc-b345-5610cb4ecce1 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:07.105369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 943/1908] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:07.720609 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62155cab-4825-4ae7-9765-c391318d265a None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:07.722094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-62155cab-4825-4ae7-9765-c391318d265a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:07.776870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62155cab-4825-4ae7-9765-c391318d265a None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:07.776870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:07.776870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:07.776870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:07.777190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] PATH_INFO: `/v3/projects/6bc379478e7c42ab8e94bcccf4688a57` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:07.777190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bc379478e7c42ab8e94bcccf4688a57)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:07.777190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:08.033381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bc379478e7c42ab8e94bcccf4688a57', 'request_id': 'req-2b51577a-932f-43e4-8a33-613d0f2329e2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:08.044507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bc379478e7c42ab8e94bcccf4688a57', 'request_id': 'req-2b51577a-932f-43e4-8a33-613d0f2329e2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:08.169327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bc379478e7c42ab8e94bcccf4688a57', 'request_id': 'req-2b51577a-932f-43e4-8a33-613d0f2329e2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:08.175538 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:08.199496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] The token cache is being invalidate because project 6bc379478e7c42ab8e94bcccf4688a57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:08.200153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:08.203133 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:08.206034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 966/1909] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:07 2022] DELETE /identity/v3/projects/6bc379478e7c42ab8e94bcccf4688a57 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:08.212361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26871f-603e-4ddc-b345-5610cb4ecce1 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:08.223197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26871f-603e-4ddc-b345-5610cb4ecce1 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:08.343810 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd201c1-9c00-4386-a165-d7b1d303d543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:08.344568 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd201c1-9c00-4386-a165-d7b1d303d543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:08.345042 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd201c1-9c00-4386-a165-d7b1d303d543 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:08.345330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd201c1-9c00-4386-a165-d7b1d303d543 admin admin] PATH_INFO: `/v3/users/6807711bf52a4f2ab5129e3c84ec437b` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:08.346571 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd201c1-9c00-4386-a165-d7b1d303d543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6807711bf52a4f2ab5129e3c84ec437b)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:08.348964 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd201c1-9c00-4386-a165-d7b1d303d543 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:08.600828 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-9cd201c1-9c00-4386-a165-d7b1d303d543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6807711bf52a4f2ab5129e3c84ec437b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:08.603105 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-9cd201c1-9c00-4386-a165-d7b1d303d543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6807711bf52a4f2ab5129e3c84ec437b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:08.604223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-9cd201c1-9c00-4386-a165-d7b1d303d543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6807711bf52a4f2ab5129e3c84ec437b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:08.624181 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-9cd201c1-9c00-4386-a165-d7b1d303d543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6807711bf52a4f2ab5129e3c84ec437b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:08.658388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-9cd201c1-9c00-4386-a165-d7b1d303d543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6807711bf52a4f2ab5129e3c84ec437b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:08.807992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 944/1910] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:08 2022] DELETE /identity/v3/users/6807711bf52a4f2ab5129e3c84ec437b => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:08.985311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:08.989570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:09.005384 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51577a-932f-43e4-8a33-613d0f2329e2 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:09.156080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b33dc8-4766-45e2-a0e7-83bae3207355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:09.161682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b33dc8-4766-45e2-a0e7-83bae3207355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:09.162247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b33dc8-4766-45e2-a0e7-83bae3207355 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:09.162750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b33dc8-4766-45e2-a0e7-83bae3207355 admin admin] PATH_INFO: `/v3/users/8ce0f05db1ba478caa48f08661bc34a3` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:09.171215 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b33dc8-4766-45e2-a0e7-83bae3207355 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ce0f05db1ba478caa48f08661bc34a3)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:09.182136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b33dc8-4766-45e2-a0e7-83bae3207355 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:09.530531 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-15b33dc8-4766-45e2-a0e7-83bae3207355 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ce0f05db1ba478caa48f08661bc34a3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:09.549644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-15b33dc8-4766-45e2-a0e7-83bae3207355 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ce0f05db1ba478caa48f08661bc34a3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:09.598093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd201c1-9c00-4386-a165-d7b1d303d543 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:09.600232 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9cd201c1-9c00-4386-a165-d7b1d303d543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:09.618763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd201c1-9c00-4386-a165-d7b1d303d543 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:09.627254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:09.628086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:09.628380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:09.628661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a admin admin] PATH_INFO: `/v3/projects/463b3f351a814862987777cbf5690160` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:09.633554 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a admin admin] RBAC: Authorizing `identity:delete_project(project_id=463b3f351a814862987777cbf5690160)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:09.637539 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:09.693541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-15b33dc8-4766-45e2-a0e7-83bae3207355 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ce0f05db1ba478caa48f08661bc34a3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:09.708992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-15b33dc8-4766-45e2-a0e7-83bae3207355 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ce0f05db1ba478caa48f08661bc34a3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:09.709682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-15b33dc8-4766-45e2-a0e7-83bae3207355 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ce0f05db1ba478caa48f08661bc34a3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:09.717250 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 967/1911] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:08 2022] DELETE /identity/v3/users/8ce0f05db1ba478caa48f08661bc34a3 => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:09.721802 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c4340-30d1-47aa-a48a-915eccb99263 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:09.722091 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c4340-30d1-47aa-a48a-915eccb99263 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:09.722571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c4340-30d1-47aa-a48a-915eccb99263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:09.746643 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-301c4340-30d1-47aa-a48a-915eccb99263 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:09.748571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-301c4340-30d1-47aa-a48a-915eccb99263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:09.856799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '463b3f351a814862987777cbf5690160', 'request_id': 'req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:09.874117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '463b3f351a814862987777cbf5690160', 'request_id': 'req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:09.877094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:09.883410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 968/1912] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:09.885100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '463b3f351a814862987777cbf5690160', 'request_id': 'req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:09.891100 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c4340-30d1-47aa-a48a-915eccb99263 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:09.892652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-301c4340-30d1-47aa-a48a-915eccb99263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:09.990785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c4340-30d1-47aa-a48a-915eccb99263 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:10.009200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a admin admin] The token cache is being invalidate because project 463b3f351a814862987777cbf5690160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:10.009809 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:10.010880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-5418b0fa-a538-4a9e-b2e8-158d52cfa33a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:10.031480 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 945/1913] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:09 2022] DELETE /identity/v3/projects/463b3f351a814862987777cbf5690160 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:10.034461 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dda800-f824-4257-ada3-871e5d9c3d21 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:10.034759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dda800-f824-4257-ada3-871e5d9c3d21 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:10.035041 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dda800-f824-4257-ada3-871e5d9c3d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:10.060258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369e45d-0dac-408e-8ce1-4f11ede533a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:10.061323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369e45d-0dac-408e-8ce1-4f11ede533a1 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:10.061487 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-46dda800-f824-4257-ada3-871e5d9c3d21 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:10.061904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369e45d-0dac-408e-8ce1-4f11ede533a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:10.062347 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369e45d-0dac-408e-8ce1-4f11ede533a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:10.062725 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-46dda800-f824-4257-ada3-871e5d9c3d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:10.064642 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5369e45d-0dac-408e-8ce1-4f11ede533a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:10.161454 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 946/1914] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:10.170662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c0282-215b-464b-b8bf-5074ca9278fa None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:10.170959 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c0282-215b-464b-b8bf-5074ca9278fa None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:10.171245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c0282-215b-464b-b8bf-5074ca9278fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:10.174225 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369e45d-0dac-408e-8ce1-4f11ede533a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:10.175936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369e45d-0dac-408e-8ce1-4f11ede533a1 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:10.188932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 969/1915] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:06:10.202988 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-e83c0282-215b-464b-b8bf-5074ca9278fa None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:10.204392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e83c0282-215b-464b-b8bf-5074ca9278fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:10.213559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 947/1916] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:10.218227 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369e45d-0dac-408e-8ce1-4f11ede533a1 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:10.229613 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369e45d-0dac-408e-8ce1-4f11ede533a1 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:10.237840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40817271-5c28-45dd-b77f-aa6b4e55db49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:10.239120 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40817271-5c28-45dd-b77f-aa6b4e55db49 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:10.239611 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40817271-5c28-45dd-b77f-aa6b4e55db49 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:10.240061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40817271-5c28-45dd-b77f-aa6b4e55db49 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:10.241120 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40817271-5c28-45dd-b77f-aa6b4e55db49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:10.242413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40817271-5c28-45dd-b77f-aa6b4e55db49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:10.244986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40817271-5c28-45dd-b77f-aa6b4e55db49 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:10.261219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 970/1917] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:06:10.269167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c0282-215b-464b-b8bf-5074ca9278fa None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:10.287244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c0282-215b-464b-b8bf-5074ca9278fa None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:10.295583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b869fe-ef2a-4e2b-ba79-3e4445ab4cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:10.296230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b869fe-ef2a-4e2b-ba79-3e4445ab4cee admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:10.296441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b869fe-ef2a-4e2b-ba79-3e4445ab4cee admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:10.296729 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b869fe-ef2a-4e2b-ba79-3e4445ab4cee admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:10.297486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b869fe-ef2a-4e2b-ba79-3e4445ab4cee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:10.299966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b869fe-ef2a-4e2b-ba79-3e4445ab4cee admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:10.676414 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 948/1918] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:10 2022] POST /identity/v3/projects => generated 377 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:10.702928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40817271-5c28-45dd-b77f-aa6b4e55db49 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:10.714548 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40817271-5c28-45dd-b77f-aa6b4e55db49 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:10.780197 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88650568-212a-4cc9-b9d1-fe4fa21c3ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:10.781560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88650568-212a-4cc9-b9d1-fe4fa21c3ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:10.785413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:10.795809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88650568-212a-4cc9-b9d1-fe4fa21c3ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:10.795809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88650568-212a-4cc9-b9d1-fe4fa21c3ac1 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:10.795809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88650568-212a-4cc9-b9d1-fe4fa21c3ac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:10.795809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88650568-212a-4cc9-b9d1-fe4fa21c3ac1 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:10.964407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 971/1919] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:10 2022] POST /identity/v3/users => generated 289 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:10.967799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b869fe-ef2a-4e2b-ba79-3e4445ab4cee admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:10.994843 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b869fe-ef2a-4e2b-ba79-3e4445ab4cee admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:11.001820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dedf76-875e-4fd0-826b-db4daee5ff2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:11.002555 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dedf76-875e-4fd0-826b-db4daee5ff2f admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:11.003402 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dedf76-875e-4fd0-826b-db4daee5ff2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:11.003730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dedf76-875e-4fd0-826b-db4daee5ff2f admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:11.004454 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dedf76-875e-4fd0-826b-db4daee5ff2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:11.006812 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dedf76-875e-4fd0-826b-db4daee5ff2f admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:11.023715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 949/1920] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:11.027346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88650568-212a-4cc9-b9d1-fe4fa21c3ac1 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:11.040904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88650568-212a-4cc9-b9d1-fe4fa21c3ac1 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:11.047227 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602ff23-970c-46db-a9ee-42e5a6a5de69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:11.048049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602ff23-970c-46db-a9ee-42e5a6a5de69 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:11.048437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602ff23-970c-46db-a9ee-42e5a6a5de69 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:11.048793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602ff23-970c-46db-a9ee-42e5a6a5de69 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:11.049641 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e602ff23-970c-46db-a9ee-42e5a6a5de69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:11.051732 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e602ff23-970c-46db-a9ee-42e5a6a5de69 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:11.066983 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 972/1921] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:11.071218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dedf76-875e-4fd0-826b-db4daee5ff2f admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:11.085018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dedf76-875e-4fd0-826b-db4daee5ff2f admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:11.093164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40c34f-d10b-47ae-8cb2-d3481d0df94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:11.093936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40c34f-d10b-47ae-8cb2-d3481d0df94c admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:11.094222 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40c34f-d10b-47ae-8cb2-d3481d0df94c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:11.094505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40c34f-d10b-47ae-8cb2-d3481d0df94c admin admin] PATH_INFO: `/v3/projects/0afc32c1f6ed4a88aac08facdbf7489f/users/82d2a0b56efd4a6a8fb77d1c18480637/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:11.127520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40c34f-d10b-47ae-8cb2-d3481d0df94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0afc32c1f6ed4a88aac08facdbf7489f, user_id=82d2a0b56efd4a6a8fb77d1c18480637, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:11.127520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40c34f-d10b-47ae-8cb2-d3481d0df94c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:11.141992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602ff23-970c-46db-a9ee-42e5a6a5de69 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:11.141992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e602ff23-970c-46db-a9ee-42e5a6a5de69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:11.155898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602ff23-970c-46db-a9ee-42e5a6a5de69 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:11.160931 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6726e39b-0cc4-4ef0-9401-688f9ce6a394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:11.161588 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726e39b-0cc4-4ef0-9401-688f9ce6a394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:11.161857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726e39b-0cc4-4ef0-9401-688f9ce6a394 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:11.162127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726e39b-0cc4-4ef0-9401-688f9ce6a394 admin admin] PATH_INFO: `/v3/users/db3c75ad89e54835934610d253194ca2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:11.168459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726e39b-0cc4-4ef0-9401-688f9ce6a394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db3c75ad89e54835934610d253194ca2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:11.171062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6726e39b-0cc4-4ef0-9401-688f9ce6a394 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:11.270339 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 950/1922] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:11 2022] PUT /identity/v3/projects/0afc32c1f6ed4a88aac08facdbf7489f/users/82d2a0b56efd4a6a8fb77d1c18480637/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:11.270339 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40c34f-d10b-47ae-8cb2-d3481d0df94c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:11.270339 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-2d40c34f-d10b-47ae-8cb2-d3481d0df94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:11.353700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40c34f-d10b-47ae-8cb2-d3481d0df94c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:11.359650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9008644-c5df-4b3d-a536-4a70d4806143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:11.359650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9008644-c5df-4b3d-a536-4a70d4806143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:11.359650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9008644-c5df-4b3d-a536-4a70d4806143 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:11.359903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9008644-c5df-4b3d-a536-4a70d4806143 admin admin] PATH_INFO: `/v3/projects/d1d1829007bb46e380944cf9e32fef49` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:11.361002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9008644-c5df-4b3d-a536-4a70d4806143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d1829007bb46e380944cf9e32fef49)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:11.362910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9008644-c5df-4b3d-a536-4a70d4806143 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:11.440829 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6726e39b-0cc4-4ef0-9401-688f9ce6a394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db3c75ad89e54835934610d253194ca2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:11.448976 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6726e39b-0cc4-4ef0-9401-688f9ce6a394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db3c75ad89e54835934610d253194ca2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:11.482545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-d9008644-c5df-4b3d-a536-4a70d4806143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d1829007bb46e380944cf9e32fef49', 'request_id': 'req-d9008644-c5df-4b3d-a536-4a70d4806143'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:11.492623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-d9008644-c5df-4b3d-a536-4a70d4806143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d1829007bb46e380944cf9e32fef49', 'request_id': 'req-d9008644-c5df-4b3d-a536-4a70d4806143'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:11.501138 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-d9008644-c5df-4b3d-a536-4a70d4806143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d1829007bb46e380944cf9e32fef49', 'request_id': 'req-d9008644-c5df-4b3d-a536-4a70d4806143'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:11.507808 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6726e39b-0cc4-4ef0-9401-688f9ce6a394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db3c75ad89e54835934610d253194ca2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:11.524221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6726e39b-0cc4-4ef0-9401-688f9ce6a394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db3c75ad89e54835934610d253194ca2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:11.524973 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6726e39b-0cc4-4ef0-9401-688f9ce6a394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db3c75ad89e54835934610d253194ca2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:11.532099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 973/1923] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:11 2022] DELETE /identity/v3/users/db3c75ad89e54835934610d253194ca2 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:11.534503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce5f0a-62c7-4b36-b718-e894c7741859 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:11.534796 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce5f0a-62c7-4b36-b718-e894c7741859 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:11.535078 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce5f0a-62c7-4b36-b718-e894c7741859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:11.577480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-73ce5f0a-62c7-4b36-b718-e894c7741859 None None] MFA Rules not processed for user `82d2a0b56efd4a6a8fb77d1c18480637`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:11.579022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-73ce5f0a-62c7-4b36-b718-e894c7741859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:11.645925 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-d9008644-c5df-4b3d-a536-4a70d4806143 admin admin] The token cache is being invalidate because project d1d1829007bb46e380944cf9e32fef49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:11.646157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-d9008644-c5df-4b3d-a536-4a70d4806143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:11.647024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-d9008644-c5df-4b3d-a536-4a70d4806143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:11.649499 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 951/1924] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:11 2022] DELETE /identity/v3/projects/d1d1829007bb46e380944cf9e32fef49 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:11.651005 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a80cf0-68e8-4eb4-810a-349c9b3c8caf None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:11.651479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a80cf0-68e8-4eb4-810a-349c9b3c8caf None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:11.651996 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a80cf0-68e8-4eb4-810a-349c9b3c8caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:11.672752 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 974/1925] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:11.674336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce5f0a-62c7-4b36-b718-e894c7741859 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:11.676168 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-a6a80cf0-68e8-4eb4-810a-349c9b3c8caf None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:11.677756 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a6a80cf0-68e8-4eb4-810a-349c9b3c8caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:11.678470 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-73ce5f0a-62c7-4b36-b718-e894c7741859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:11.750789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 952/1926] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:11.755291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a80cf0-68e8-4eb4-810a-349c9b3c8caf None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:11.756752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a6a80cf0-68e8-4eb4-810a-349c9b3c8caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:11.761988 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce5f0a-62c7-4b36-b718-e894c7741859 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:11.767383 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf4cfd-5287-476e-a1a9-00356933c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:11.768219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf4cfd-5287-476e-a1a9-00356933c7e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:11.768619 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf4cfd-5287-476e-a1a9-00356933c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:11.768990 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf4cfd-5287-476e-a1a9-00356933c7e6 admin admin] PATH_INFO: `/v3/users/39939f217b3544dfa62cd94944a17052` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:11.770473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf4cfd-5287-476e-a1a9-00356933c7e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39939f217b3544dfa62cd94944a17052)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:11.771064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a80cf0-68e8-4eb4-810a-349c9b3c8caf None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:11.772695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf4cfd-5287-476e-a1a9-00356933c7e6 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:11.778693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a88386-2d28-4fb7-a80a-2978a9f43d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:11.782266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a88386-2d28-4fb7-a80a-2978a9f43d0e admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:11.782266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a88386-2d28-4fb7-a80a-2978a9f43d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:11.782266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a88386-2d28-4fb7-a80a-2978a9f43d0e admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:11.782266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a88386-2d28-4fb7-a80a-2978a9f43d0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:11.782756 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a88386-2d28-4fb7-a80a-2978a9f43d0e admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:11.915846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 953/1927] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:11 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:11.915846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a88386-2d28-4fb7-a80a-2978a9f43d0e admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:11.917356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b2a88386-2d28-4fb7-a80a-2978a9f43d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:11.983192 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a88386-2d28-4fb7-a80a-2978a9f43d0e admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:11.988863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfede501-496e-447d-837a-2ee37aa7ac15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:11.989498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfede501-496e-447d-837a-2ee37aa7ac15 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:11.989917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfede501-496e-447d-837a-2ee37aa7ac15 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:11.990238 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfede501-496e-447d-837a-2ee37aa7ac15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:12.000266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-cdaf4cfd-5287-476e-a1a9-00356933c7e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39939f217b3544dfa62cd94944a17052'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:12.015264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-cdaf4cfd-5287-476e-a1a9-00356933c7e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39939f217b3544dfa62cd94944a17052'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:12.066214 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfede501-496e-447d-837a-2ee37aa7ac15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:12.066214 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfede501-496e-447d-837a-2ee37aa7ac15 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:12.074889 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 954/1928] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:12.075826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfede501-496e-447d-837a-2ee37aa7ac15 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:12.086737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfede501-496e-447d-837a-2ee37aa7ac15 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:12.092050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364e144-fb1c-4757-81f2-7515aad1a350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:12.092749 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364e144-fb1c-4757-81f2-7515aad1a350 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:12.093098 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364e144-fb1c-4757-81f2-7515aad1a350 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:12.093484 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364e144-fb1c-4757-81f2-7515aad1a350 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:12.094246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364e144-fb1c-4757-81f2-7515aad1a350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:12.096257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364e144-fb1c-4757-81f2-7515aad1a350 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:12.156429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-cdaf4cfd-5287-476e-a1a9-00356933c7e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39939f217b3544dfa62cd94944a17052'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:12.172064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-cdaf4cfd-5287-476e-a1a9-00356933c7e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39939f217b3544dfa62cd94944a17052'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:12.172759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-cdaf4cfd-5287-476e-a1a9-00356933c7e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39939f217b3544dfa62cd94944a17052'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:12.179707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 975/1929] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:11 2022] DELETE /identity/v3/users/39939f217b3544dfa62cd94944a17052 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:12.209625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 955/1930] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:12 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:12.216684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf4cfd-5287-476e-a1a9-00356933c7e6 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:12.223191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-cdaf4cfd-5287-476e-a1a9-00356933c7e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:12.357326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf4cfd-5287-476e-a1a9-00356933c7e6 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:12.364716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a0f38-b62c-479b-b5ba-d045f25fa61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:12.365652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a0f38-b62c-479b-b5ba-d045f25fa61e admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:12.366244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a0f38-b62c-479b-b5ba-d045f25fa61e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:12.368142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a0f38-b62c-479b-b5ba-d045f25fa61e admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:12.368670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a0f38-b62c-479b-b5ba-d045f25fa61e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:12.369057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:12.371230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a0f38-b62c-479b-b5ba-d045f25fa61e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:12.387802 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 976/1931] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:12.389321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364e144-fb1c-4757-81f2-7515aad1a350 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:12.401017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364e144-fb1c-4757-81f2-7515aad1a350 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:12.407924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0e2da-e304-4423-b72e-ea2353282d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:12.408569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0e2da-e304-4423-b72e-ea2353282d00 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:12.408952 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0e2da-e304-4423-b72e-ea2353282d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:12.409266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0e2da-e304-4423-b72e-ea2353282d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:12.409933 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0e2da-e304-4423-b72e-ea2353282d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:12.411754 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0e2da-e304-4423-b72e-ea2353282d00 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:12.429863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 956/1932] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:06:12.430332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a0f38-b62c-479b-b5ba-d045f25fa61e admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:12.440443 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a0f38-b62c-479b-b5ba-d045f25fa61e admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:12.445848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91b696-d64b-4704-b1bf-68305f9edd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:12.446565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91b696-d64b-4704-b1bf-68305f9edd8e admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:12.446788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91b696-d64b-4704-b1bf-68305f9edd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:12.447157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91b696-d64b-4704-b1bf-68305f9edd8e admin admin] PATH_INFO: `/v3/projects/06bfdd1e85394a89a138b62bb99bb9f5/users/e2e6e7c7dcba47468962695d804a46bc/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:12.467244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91b696-d64b-4704-b1bf-68305f9edd8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=06bfdd1e85394a89a138b62bb99bb9f5, user_id=e2e6e7c7dcba47468962695d804a46bc, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:12.470141 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91b696-d64b-4704-b1bf-68305f9edd8e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:12.572565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 977/1933] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:12 2022] PUT /identity/v3/projects/06bfdd1e85394a89a138b62bb99bb9f5/users/e2e6e7c7dcba47468962695d804a46bc/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:12.583424 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2a0e6-69f9-450d-a10e-183b7237cfb1 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:12.584474 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2a0e6-69f9-450d-a10e-183b7237cfb1 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:12.585128 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2a0e6-69f9-450d-a10e-183b7237cfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:12.610517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91b696-d64b-4704-b1bf-68305f9edd8e admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:12.614612 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5a91b696-d64b-4704-b1bf-68305f9edd8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:12.643865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-21b2a0e6-69f9-450d-a10e-183b7237cfb1 None None] MFA Rules not processed for user `e2e6e7c7dcba47468962695d804a46bc`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:12.645788 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-21b2a0e6-69f9-450d-a10e-183b7237cfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:12.676481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:12.742666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91b696-d64b-4704-b1bf-68305f9edd8e admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:12.745714 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 957/1934] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:12.747428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2a0e6-69f9-450d-a10e-183b7237cfb1 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:12.750296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-21b2a0e6-69f9-450d-a10e-183b7237cfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:12.752909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:12.754270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:12.754556 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:12.754838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] PATH_INFO: `/v3/projects/34130fec4ec3442285c9c4327531b33e` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:12.756172 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34130fec4ec3442285c9c4327531b33e)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:12.758202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:12.771032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2a0e6-69f9-450d-a10e-183b7237cfb1 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:12.779218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f21d1-fa43-4c94-97bc-7a8e8e6bfda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:12.783383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f21d1-fa43-4c94-97bc-7a8e8e6bfda2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:12.783383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f21d1-fa43-4c94-97bc-7a8e8e6bfda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:12.783383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f21d1-fa43-4c94-97bc-7a8e8e6bfda2 admin admin] PATH_INFO: `/v3/users/6537f456e0254b558c4d304e55f39d70` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:12.783383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f21d1-fa43-4c94-97bc-7a8e8e6bfda2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6537f456e0254b558c4d304e55f39d70)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:12.784354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f21d1-fa43-4c94-97bc-7a8e8e6bfda2 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:12.893331 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34130fec4ec3442285c9c4327531b33e', 'request_id': 'req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:12.902934 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34130fec4ec3442285c9c4327531b33e', 'request_id': 'req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.014116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34130fec4ec3442285c9c4327531b33e', 'request_id': 'req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.029932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] The token cache is being invalidate because project 34130fec4ec3442285c9c4327531b33e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:13.030534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.031539 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.037447 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 978/1935] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:12 2022] DELETE /identity/v3/projects/34130fec4ec3442285c9c4327531b33e => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:13.039273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:13.041225 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:13.086552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-911f21d1-fa43-4c94-97bc-7a8e8e6bfda2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6537f456e0254b558c4d304e55f39d70'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.086552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-911f21d1-fa43-4c94-97bc-7a8e8e6bfda2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6537f456e0254b558c4d304e55f39d70'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.086552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-911f21d1-fa43-4c94-97bc-7a8e8e6bfda2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6537f456e0254b558c4d304e55f39d70'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.120701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-911f21d1-fa43-4c94-97bc-7a8e8e6bfda2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6537f456e0254b558c4d304e55f39d70'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.152094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-911f21d1-fa43-4c94-97bc-7a8e8e6bfda2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6537f456e0254b558c4d304e55f39d70'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.159005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ae347-1ffe-4fdf-9f14-a35a957bf079 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:13.169832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee653cc9-eca9-483c-acdb-01ea07b6c65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:13.170756 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee653cc9-eca9-483c-acdb-01ea07b6c65c admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:13.171155 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee653cc9-eca9-483c-acdb-01ea07b6c65c admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:13.171511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee653cc9-eca9-483c-acdb-01ea07b6c65c admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:13.177954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee653cc9-eca9-483c-acdb-01ea07b6c65c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:13.177954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee653cc9-eca9-483c-acdb-01ea07b6c65c admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:13.275198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 958/1936] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:12 2022] DELETE /identity/v3/users/6537f456e0254b558c4d304e55f39d70 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:13.276148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f21d1-fa43-4c94-97bc-7a8e8e6bfda2 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:13.278230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-911f21d1-fa43-4c94-97bc-7a8e8e6bfda2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:13.367382 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f21d1-fa43-4c94-97bc-7a8e8e6bfda2 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:13.376007 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:13.377179 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:13.377777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:13.378065 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c admin admin] PATH_INFO: `/v3/projects/7066fcb933a84548b8b2684ba5e7c98f` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:13.380169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7066fcb933a84548b8b2684ba5e7c98f)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:13.384752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:13.449929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:13.451686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 979/1937] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:13 2022] POST /identity/v3/projects => generated 377 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:13.452782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee653cc9-eca9-483c-acdb-01ea07b6c65c admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:13.456055 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ee653cc9-eca9-483c-acdb-01ea07b6c65c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:13.544140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7066fcb933a84548b8b2684ba5e7c98f', 'request_id': 'req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.551577 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7066fcb933a84548b8b2684ba5e7c98f', 'request_id': 'req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.555043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee653cc9-eca9-483c-acdb-01ea07b6c65c admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:13.559558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7066fcb933a84548b8b2684ba5e7c98f', 'request_id': 'req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.639193 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c admin admin] The token cache is being invalidate because project 7066fcb933a84548b8b2684ba5e7c98f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:13.639874 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.643005 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e40c73c7-15b2-4043-b8d1-e77d9258ff2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.645842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 959/1938] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:13 2022] DELETE /identity/v3/projects/7066fcb933a84548b8b2684ba5e7c98f => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:13.646950 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76068854-0df8-4c1e-a157-5d42554a6e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:13.647434 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e708de-26ab-48e1-a0df-1a96b8cd89ca None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:13.648327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e708de-26ab-48e1-a0df-1a96b8cd89ca None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:13.648597 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e708de-26ab-48e1-a0df-1a96b8cd89ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:13.648766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76068854-0df8-4c1e-a157-5d42554a6e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:13.649026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76068854-0df8-4c1e-a157-5d42554a6e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:13.649562 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76068854-0df8-4c1e-a157-5d42554a6e51 admin admin] PATH_INFO: `/v3/users/fbf6e928e6414b9c9b7f3b5b076fb2e1` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:13.652281 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76068854-0df8-4c1e-a157-5d42554a6e51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbf6e928e6414b9c9b7f3b5b076fb2e1)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:13.657551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76068854-0df8-4c1e-a157-5d42554a6e51 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:13.673090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-82e708de-26ab-48e1-a0df-1a96b8cd89ca None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:13.674371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-82e708de-26ab-48e1-a0df-1a96b8cd89ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:13.685762 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 960/1939] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:13.686761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e708de-26ab-48e1-a0df-1a96b8cd89ca None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:13.688425 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-82e708de-26ab-48e1-a0df-1a96b8cd89ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:13.707893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e708de-26ab-48e1-a0df-1a96b8cd89ca None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:13.724945 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15161cbd-488b-434a-8736-9c47d0b4b3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:13.725606 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15161cbd-488b-434a-8736-9c47d0b4b3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:13.726563 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15161cbd-488b-434a-8736-9c47d0b4b3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:13.727004 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15161cbd-488b-434a-8736-9c47d0b4b3d4 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:13.728584 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15161cbd-488b-434a-8736-9c47d0b4b3d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:13.736621 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15161cbd-488b-434a-8736-9c47d0b4b3d4 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:13.932146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 961/1940] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:13 2022] POST /identity/v3/users => generated 295 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:13.933186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e588ec-095a-441d-b9be-c57e650d1752 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:13.933819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e588ec-095a-441d-b9be-c57e650d1752 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:13.935269 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e588ec-095a-441d-b9be-c57e650d1752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:13.941237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:13.958494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-76068854-0df8-4c1e-a157-5d42554a6e51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbf6e928e6414b9c9b7f3b5b076fb2e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.971375 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-11e588ec-095a-441d-b9be-c57e650d1752 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:13.974044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-11e588ec-095a-441d-b9be-c57e650d1752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:13.977524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-76068854-0df8-4c1e-a157-5d42554a6e51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbf6e928e6414b9c9b7f3b5b076fb2e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:13.987853 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 962/1941] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:13.988912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e588ec-095a-441d-b9be-c57e650d1752 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:13.994618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-11e588ec-095a-441d-b9be-c57e650d1752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:14.059372 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-76068854-0df8-4c1e-a157-5d42554a6e51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbf6e928e6414b9c9b7f3b5b076fb2e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:14.063623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e588ec-095a-441d-b9be-c57e650d1752 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:14.070341 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5c103-6f3f-4247-9a78-2be91d8228b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:14.071333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5c103-6f3f-4247-9a78-2be91d8228b7 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:14.071875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5c103-6f3f-4247-9a78-2be91d8228b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:14.072331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5c103-6f3f-4247-9a78-2be91d8228b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:14.077187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-76068854-0df8-4c1e-a157-5d42554a6e51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbf6e928e6414b9c9b7f3b5b076fb2e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:14.078314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-76068854-0df8-4c1e-a157-5d42554a6e51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbf6e928e6414b9c9b7f3b5b076fb2e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:14.087255 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5c103-6f3f-4247-9a78-2be91d8228b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:14.088641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5c103-6f3f-4247-9a78-2be91d8228b7 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:14.089384 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 980/1942] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:13 2022] DELETE /identity/v3/users/fbf6e928e6414b9c9b7f3b5b076fb2e1 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:14.092417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76068854-0df8-4c1e-a157-5d42554a6e51 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:14.097841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-76068854-0df8-4c1e-a157-5d42554a6e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:14.108564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 963/1943] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:14.109987 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068ec2a-afd2-4be7-809a-d9e87324efc8 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:14.110293 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068ec2a-afd2-4be7-809a-d9e87324efc8 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:14.110581 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068ec2a-afd2-4be7-809a-d9e87324efc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:14.142790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-7068ec2a-afd2-4be7-809a-d9e87324efc8 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:14.144141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-7068ec2a-afd2-4be7-809a-d9e87324efc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:14.225119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76068854-0df8-4c1e-a157-5d42554a6e51 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:14.232266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224d9fd-3910-4ff2-9c36-47df3929c729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:14.232266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d9fd-3910-4ff2-9c36-47df3929c729 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:14.240577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d9fd-3910-4ff2-9c36-47df3929c729 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:14.240577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d9fd-3910-4ff2-9c36-47df3929c729 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:14.240577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224d9fd-3910-4ff2-9c36-47df3929c729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:14.240577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224d9fd-3910-4ff2-9c36-47df3929c729 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:14.251435 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 981/1944] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:14.256513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224d9fd-3910-4ff2-9c36-47df3929c729 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:14.268801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 964/1945] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:14.270637 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224d9fd-3910-4ff2-9c36-47df3929c729 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:14.274564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068ec2a-afd2-4be7-809a-d9e87324efc8 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:14.282985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b31a1-5ecf-468c-a26a-62ef8ed0ff17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:14.288017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b31a1-5ecf-468c-a26a-62ef8ed0ff17 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:14.288017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b31a1-5ecf-468c-a26a-62ef8ed0ff17 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:14.288017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b31a1-5ecf-468c-a26a-62ef8ed0ff17 admin admin] PATH_INFO: `/v3/projects/4ce09d7520e9436e83b97433a464729b/users/fa6ed4f8ba084ead9ffd3f09da3dafbe/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:14.295795 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068ec2a-afd2-4be7-809a-d9e87324efc8 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:14.303952 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ec21a-c23e-4e49-91c7-15895b88b874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:14.306963 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ec21a-c23e-4e49-91c7-15895b88b874 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:14.311043 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ec21a-c23e-4e49-91c7-15895b88b874 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:14.311187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b31a1-5ecf-468c-a26a-62ef8ed0ff17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ce09d7520e9436e83b97433a464729b, user_id=fa6ed4f8ba084ead9ffd3f09da3dafbe, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:14.311286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ec21a-c23e-4e49-91c7-15895b88b874 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:14.311286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ec21a-c23e-4e49-91c7-15895b88b874 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:14.311286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ec21a-c23e-4e49-91c7-15895b88b874 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:14.311286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ec21a-c23e-4e49-91c7-15895b88b874 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:14.311618 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b31a1-5ecf-468c-a26a-62ef8ed0ff17 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:14.328381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 965/1946] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:06:14.332980 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ec21a-c23e-4e49-91c7-15895b88b874 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:14.346771 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ec21a-c23e-4e49-91c7-15895b88b874 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:14.353167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645468a-decd-4ca3-9777-c8a999860d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:14.354005 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645468a-decd-4ca3-9777-c8a999860d1a admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:14.354488 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645468a-decd-4ca3-9777-c8a999860d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:14.354833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645468a-decd-4ca3-9777-c8a999860d1a admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:14.355615 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9645468a-decd-4ca3-9777-c8a999860d1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:14.357604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9645468a-decd-4ca3-9777-c8a999860d1a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:14.443943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 982/1947] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:14 2022] PUT /identity/v3/projects/4ce09d7520e9436e83b97433a464729b/users/fa6ed4f8ba084ead9ffd3f09da3dafbe/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:14.448472 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b31a1-5ecf-468c-a26a-62ef8ed0ff17 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:14.473039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 966/1948] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:14 2022] POST /identity/v3/projects => generated 363 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:06:14.481028 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645468a-decd-4ca3-9777-c8a999860d1a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:14.544338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b31a1-5ecf-468c-a26a-62ef8ed0ff17 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:14.571034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645468a-decd-4ca3-9777-c8a999860d1a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:14.576101 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c99c5e-9776-4804-a4a0-a39785dfc9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:14.576738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c99c5e-9776-4804-a4a0-a39785dfc9b1 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:14.577023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c99c5e-9776-4804-a4a0-a39785dfc9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:14.577302 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c99c5e-9776-4804-a4a0-a39785dfc9b1 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:14.578008 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c99c5e-9776-4804-a4a0-a39785dfc9b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:14.580026 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c99c5e-9776-4804-a4a0-a39785dfc9b1 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:14.619245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff6021-e597-4286-9c79-675bb7c52faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:14.620607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff6021-e597-4286-9c79-675bb7c52faa admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:14.620607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff6021-e597-4286-9c79-675bb7c52faa admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:14.620607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff6021-e597-4286-9c79-675bb7c52faa admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:14.621862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff6021-e597-4286-9c79-675bb7c52faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:14.624155 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff6021-e597-4286-9c79-675bb7c52faa admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:14.636473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:14.643650 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 983/1949] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:06:14.646378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff6021-e597-4286-9c79-675bb7c52faa admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:14.661170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff6021-e597-4286-9c79-675bb7c52faa admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:14.667077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 967/1950] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:14 2022] POST /identity/v3/users => generated 282 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:14.677515 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c99c5e-9776-4804-a4a0-a39785dfc9b1 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:14.688399 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563a1ca-2ab9-43c4-a2d6-a626569421dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:14.689288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563a1ca-2ab9-43c4-a2d6-a626569421dd admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:14.689742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563a1ca-2ab9-43c4-a2d6-a626569421dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:14.690992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563a1ca-2ab9-43c4-a2d6-a626569421dd admin admin] PATH_INFO: `/v3/projects/4ce09d7520e9436e83b97433a464729b/users/fa6ed4f8ba084ead9ffd3f09da3dafbe/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:14.693601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563a1ca-2ab9-43c4-a2d6-a626569421dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ce09d7520e9436e83b97433a464729b, user_id=fa6ed4f8ba084ead9ffd3f09da3dafbe, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:14.699143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c99c5e-9776-4804-a4a0-a39785dfc9b1 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:14.699651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563a1ca-2ab9-43c4-a2d6-a626569421dd admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:14.723684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 984/1951] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:14 2022] PUT /identity/v3/projects/4ce09d7520e9436e83b97433a464729b/users/fa6ed4f8ba084ead9ffd3f09da3dafbe/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:14.725637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e040d-5734-4b53-bbc5-7ffc1913d178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:14.726304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e040d-5734-4b53-bbc5-7ffc1913d178 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:14.726596 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e040d-5734-4b53-bbc5-7ffc1913d178 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:14.726875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e040d-5734-4b53-bbc5-7ffc1913d178 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:14.727653 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e040d-5734-4b53-bbc5-7ffc1913d178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:14.729146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa3865-7564-49e6-b340-50a342b7b7e6 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:14.729347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e040d-5734-4b53-bbc5-7ffc1913d178 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:14.729454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa3865-7564-49e6-b340-50a342b7b7e6 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:14.729699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa3865-7564-49e6-b340-50a342b7b7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:14.747323 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 968/1952] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:14.748435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e040d-5734-4b53-bbc5-7ffc1913d178 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:14.759859 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e040d-5734-4b53-bbc5-7ffc1913d178 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:14.765860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-27aa3865-7564-49e6-b340-50a342b7b7e6 None None] MFA Rules not processed for user `fa6ed4f8ba084ead9ffd3f09da3dafbe`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:14.767330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-27aa3865-7564-49e6-b340-50a342b7b7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:14.813193 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:14.831383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cecd38-ce8c-40c7-bbc9-ef245f8d5ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:14.835509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cecd38-ce8c-40c7-bbc9-ef245f8d5ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:14.835509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cecd38-ce8c-40c7-bbc9-ef245f8d5ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:14.835509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cecd38-ce8c-40c7-bbc9-ef245f8d5ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:14.835509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cecd38-ce8c-40c7-bbc9-ef245f8d5ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:14.836146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cecd38-ce8c-40c7-bbc9-ef245f8d5ed3 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:14.854054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 985/1953] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:14.857715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 969/1954] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:06:14.869678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa3865-7564-49e6-b340-50a342b7b7e6 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:14.869831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cecd38-ce8c-40c7-bbc9-ef245f8d5ed3 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:14.869831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-77cecd38-ce8c-40c7-bbc9-ef245f8d5ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:14.877035 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa3865-7564-49e6-b340-50a342b7b7e6 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:14.882779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c494f-f208-493d-b919-77b50d48ae40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:14.883958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c494f-f208-493d-b919-77b50d48ae40 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:14.884304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c494f-f208-493d-b919-77b50d48ae40 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:14.884679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c494f-f208-493d-b919-77b50d48ae40 admin admin] PATH_INFO: `/v3/projects/71732c42b01744839e51ba6bd3af2e16/users/5288337071f74459acf4fb1d5abb7d07/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:14.900206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c494f-f208-493d-b919-77b50d48ae40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71732c42b01744839e51ba6bd3af2e16, user_id=5288337071f74459acf4fb1d5abb7d07, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:14.903079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c494f-f208-493d-b919-77b50d48ae40 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:14.928659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cecd38-ce8c-40c7-bbc9-ef245f8d5ed3 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:14.933850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb70f3-67f4-4448-adcb-3626fdb575d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:14.934508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb70f3-67f4-4448-adcb-3626fdb575d3 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:14.934791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb70f3-67f4-4448-adcb-3626fdb575d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:14.935065 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb70f3-67f4-4448-adcb-3626fdb575d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:14.944347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb70f3-67f4-4448-adcb-3626fdb575d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:14.946556 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb70f3-67f4-4448-adcb-3626fdb575d3 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:14.959829 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 970/1955] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:06:14.960778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb70f3-67f4-4448-adcb-3626fdb575d3 service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:14.962114 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8dfb70f3-67f4-4448-adcb-3626fdb575d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:14.977587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb70f3-67f4-4448-adcb-3626fdb575d3 service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:14.997533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 986/1956] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:14 2022] PUT /identity/v3/projects/71732c42b01744839e51ba6bd3af2e16/users/5288337071f74459acf4fb1d5abb7d07/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:15.003407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c494f-f208-493d-b919-77b50d48ae40 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:15.007631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c97c494f-f208-493d-b919-77b50d48ae40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:15.057835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4674db-cfec-4174-ba37-5e22e9768710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:15.058488 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4674db-cfec-4174-ba37-5e22e9768710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:15.058765 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4674db-cfec-4174-ba37-5e22e9768710 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:15.059035 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4674db-cfec-4174-ba37-5e22e9768710 admin admin] PATH_INFO: `/v3/projects/37a36b3e3284428a849f6b4e8cf26b11` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:15.060256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4674db-cfec-4174-ba37-5e22e9768710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37a36b3e3284428a849f6b4e8cf26b11)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:15.062217 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4674db-cfec-4174-ba37-5e22e9768710 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:15.086921 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c494f-f208-493d-b919-77b50d48ae40 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:15.092022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:15.092675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:15.092980 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:15.093361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] PATH_INFO: `/v3/projects/6aa80fa7719448ceb753335264aca52a` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:15.094487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aa80fa7719448ceb753335264aca52a)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:15.096431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:15.185009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ee4674db-cfec-4174-ba37-5e22e9768710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37a36b3e3284428a849f6b4e8cf26b11', 'request_id': 'req-ee4674db-cfec-4174-ba37-5e22e9768710'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.190936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ee4674db-cfec-4174-ba37-5e22e9768710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37a36b3e3284428a849f6b4e8cf26b11', 'request_id': 'req-ee4674db-cfec-4174-ba37-5e22e9768710'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.197965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ee4674db-cfec-4174-ba37-5e22e9768710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37a36b3e3284428a849f6b4e8cf26b11', 'request_id': 'req-ee4674db-cfec-4174-ba37-5e22e9768710'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.217575 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aa80fa7719448ceb753335264aca52a', 'request_id': 'req-7fc1e941-b17b-444f-b689-a48255ec9a39'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.223582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aa80fa7719448ceb753335264aca52a', 'request_id': 'req-7fc1e941-b17b-444f-b689-a48255ec9a39'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.351160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ee4674db-cfec-4174-ba37-5e22e9768710 admin admin] The token cache is being invalidate because project 37a36b3e3284428a849f6b4e8cf26b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:15.351451 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ee4674db-cfec-4174-ba37-5e22e9768710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.352166 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ee4674db-cfec-4174-ba37-5e22e9768710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.353838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 971/1957] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:14 2022] DELETE /identity/v3/projects/37a36b3e3284428a849f6b4e8cf26b11 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:15.355109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1364c9-4dc7-48ec-b95a-22a072234b6a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:15.355784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1364c9-4dc7-48ec-b95a-22a072234b6a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:15.356120 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1364c9-4dc7-48ec-b95a-22a072234b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:15.386902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ab1364c9-4dc7-48ec-b95a-22a072234b6a None None] MFA Rules not processed for user `5288337071f74459acf4fb1d5abb7d07`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:15.388186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ab1364c9-4dc7-48ec-b95a-22a072234b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:15.401571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aa80fa7719448ceb753335264aca52a', 'request_id': 'req-7fc1e941-b17b-444f-b689-a48255ec9a39'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.415552 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] The token cache is being invalidate because project 6aa80fa7719448ceb753335264aca52a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:15.415835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.416496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.418403 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 987/1958] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:15 2022] DELETE /identity/v3/projects/6aa80fa7719448ceb753335264aca52a => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:15.422085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:15.424822 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:15.463015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 972/1959] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:15.464182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1364c9-4dc7-48ec-b95a-22a072234b6a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:15.467966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ab1364c9-4dc7-48ec-b95a-22a072234b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:15.491076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1e941-b17b-444f-b689-a48255ec9a39 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:15.495850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b682edb-5a3b-4408-a3e4-17e5bfd8b192 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:15.496492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b682edb-5a3b-4408-a3e4-17e5bfd8b192 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:15.496817 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b682edb-5a3b-4408-a3e4-17e5bfd8b192 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:15.497501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b682edb-5a3b-4408-a3e4-17e5bfd8b192 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:15.499435 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2b682edb-5a3b-4408-a3e4-17e5bfd8b192 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:15.540445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1364c9-4dc7-48ec-b95a-22a072234b6a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:15.545738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde3067-f6a9-4544-94bf-56c1ae9f98e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:15.546775 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde3067-f6a9-4544-94bf-56c1ae9f98e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:15.546775 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde3067-f6a9-4544-94bf-56c1ae9f98e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:15.546981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde3067-f6a9-4544-94bf-56c1ae9f98e1 admin admin] PATH_INFO: `/v3/users/c989d519e87d4e4bb3ccd8c5b6fca8a0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:15.548225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde3067-f6a9-4544-94bf-56c1ae9f98e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c989d519e87d4e4bb3ccd8c5b6fca8a0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:15.550169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde3067-f6a9-4544-94bf-56c1ae9f98e1 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:15.568307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b682edb-5a3b-4408-a3e4-17e5bfd8b192 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:15.569870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b682edb-5a3b-4408-a3e4-17e5bfd8b192 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:15.579909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 988/1960] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:15.580775 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b682edb-5a3b-4408-a3e4-17e5bfd8b192 service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:15.582119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2b682edb-5a3b-4408-a3e4-17e5bfd8b192 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:15.597156 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b682edb-5a3b-4408-a3e4-17e5bfd8b192 service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:15.603674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32a418-1554-4dde-8739-e3509e1e7f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:15.604456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32a418-1554-4dde-8739-e3509e1e7f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:15.604762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32a418-1554-4dde-8739-e3509e1e7f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:15.605039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32a418-1554-4dde-8739-e3509e1e7f2b admin admin] PATH_INFO: `/v3/users/2066a43ee12b4c168af072b34711047f` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:15.606349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32a418-1554-4dde-8739-e3509e1e7f2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2066a43ee12b4c168af072b34711047f)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:15.608387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32a418-1554-4dde-8739-e3509e1e7f2b admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:15.798874 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4bde3067-f6a9-4544-94bf-56c1ae9f98e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c989d519e87d4e4bb3ccd8c5b6fca8a0'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.799610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4bde3067-f6a9-4544-94bf-56c1ae9f98e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c989d519e87d4e4bb3ccd8c5b6fca8a0'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.800225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4bde3067-f6a9-4544-94bf-56c1ae9f98e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c989d519e87d4e4bb3ccd8c5b6fca8a0'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.808580 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4bde3067-f6a9-4544-94bf-56c1ae9f98e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c989d519e87d4e4bb3ccd8c5b6fca8a0'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.809555 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:15.830670 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4bde3067-f6a9-4544-94bf-56c1ae9f98e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c989d519e87d4e4bb3ccd8c5b6fca8a0'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.903384 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-ca32a418-1554-4dde-8739-e3509e1e7f2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2066a43ee12b4c168af072b34711047f'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.910938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-ca32a418-1554-4dde-8739-e3509e1e7f2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2066a43ee12b4c168af072b34711047f'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:15.929871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 973/1961] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:15 2022] DELETE /identity/v3/users/c989d519e87d4e4bb3ccd8c5b6fca8a0 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:15.930804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde3067-f6a9-4544-94bf-56c1ae9f98e1 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:15.932479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4bde3067-f6a9-4544-94bf-56c1ae9f98e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:16.010175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde3067-f6a9-4544-94bf-56c1ae9f98e1 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:16.017142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af9c6-ccb2-4fcf-a44d-f9d0458c7a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:16.017787 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af9c6-ccb2-4fcf-a44d-f9d0458c7a1c admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:16.018072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af9c6-ccb2-4fcf-a44d-f9d0458c7a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:16.018353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af9c6-ccb2-4fcf-a44d-f9d0458c7a1c admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:16.019040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8af9c6-ccb2-4fcf-a44d-f9d0458c7a1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:16.021060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8af9c6-ccb2-4fcf-a44d-f9d0458c7a1c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:16.049071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-ca32a418-1554-4dde-8739-e3509e1e7f2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2066a43ee12b4c168af072b34711047f'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:16.063322 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-ca32a418-1554-4dde-8739-e3509e1e7f2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2066a43ee12b4c168af072b34711047f'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:16.064195 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-ca32a418-1554-4dde-8739-e3509e1e7f2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2066a43ee12b4c168af072b34711047f'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:16.071065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 989/1962] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:15 2022] DELETE /identity/v3/users/2066a43ee12b4c168af072b34711047f => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:16.146233 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 974/1963] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:15 2022] POST /identity/v3/projects => generated 363 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:16.147632 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32a418-1554-4dde-8739-e3509e1e7f2b admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:16.151504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ca32a418-1554-4dde-8739-e3509e1e7f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:16.189231 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:16.240691 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32a418-1554-4dde-8739-e3509e1e7f2b admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:16.320514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551813e-10ac-4c6e-950f-7fc4c18ca94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:16.320514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551813e-10ac-4c6e-950f-7fc4c18ca94f admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:16.320514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551813e-10ac-4c6e-950f-7fc4c18ca94f admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:16.320514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551813e-10ac-4c6e-950f-7fc4c18ca94f admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:16.321256 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551813e-10ac-4c6e-950f-7fc4c18ca94f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:16.323884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551813e-10ac-4c6e-950f-7fc4c18ca94f admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:16.457672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 990/1964] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:16 2022] POST /identity/v3/users => generated 288 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:16.462570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af9c6-ccb2-4fcf-a44d-f9d0458c7a1c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:16.478577 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af9c6-ccb2-4fcf-a44d-f9d0458c7a1c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:16.485663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf40982-26d4-46b3-87b6-414a6fa79fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:16.486852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf40982-26d4-46b3-87b6-414a6fa79fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:16.487227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf40982-26d4-46b3-87b6-414a6fa79fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:16.487591 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf40982-26d4-46b3-87b6-414a6fa79fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:16.488824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf40982-26d4-46b3-87b6-414a6fa79fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:16.490558 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf40982-26d4-46b3-87b6-414a6fa79fb1 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:16.503103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 975/1965] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:16.508176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551813e-10ac-4c6e-950f-7fc4c18ca94f admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:16.520880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551813e-10ac-4c6e-950f-7fc4c18ca94f admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:16.526691 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1424084-c4e7-4fb3-8c15-0aa6be2c7038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:16.527395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1424084-c4e7-4fb3-8c15-0aa6be2c7038 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:16.527722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1424084-c4e7-4fb3-8c15-0aa6be2c7038 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:16.528004 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1424084-c4e7-4fb3-8c15-0aa6be2c7038 admin admin] PATH_INFO: `/v3/projects/4ea9dc61b45c433c9910dc767ac22873/users/88a3757553a64337a95c74636c0aab8e/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:16.547920 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1424084-c4e7-4fb3-8c15-0aa6be2c7038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea9dc61b45c433c9910dc767ac22873, user_id=88a3757553a64337a95c74636c0aab8e, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:16.550718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1424084-c4e7-4fb3-8c15-0aa6be2c7038 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:16.570252 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf40982-26d4-46b3-87b6-414a6fa79fb1 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:16.572146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-bbf40982-26d4-46b3-87b6-414a6fa79fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:16.585926 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf40982-26d4-46b3-87b6-414a6fa79fb1 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:16.591792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:16.592949 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:16.593135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:16.593277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] PATH_INFO: `/v3/projects/65ab32d45feb4576afc7be0fb6fece83` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:16.594586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] RBAC: Authorizing `identity:delete_project(project_id=65ab32d45feb4576afc7be0fb6fece83)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:16.596933 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:16.650449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 991/1966] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:16 2022] PUT /identity/v3/projects/4ea9dc61b45c433c9910dc767ac22873/users/88a3757553a64337a95c74636c0aab8e/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:16.654827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1424084-c4e7-4fb3-8c15-0aa6be2c7038 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:16.734736 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1424084-c4e7-4fb3-8c15-0aa6be2c7038 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:16.748518 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65ab32d45feb4576afc7be0fb6fece83', 'request_id': 'req-45e8d42a-2170-47db-88b6-b0822de1604d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:16.756816 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65ab32d45feb4576afc7be0fb6fece83', 'request_id': 'req-45e8d42a-2170-47db-88b6-b0822de1604d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:16.764444 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65ab32d45feb4576afc7be0fb6fece83', 'request_id': 'req-45e8d42a-2170-47db-88b6-b0822de1604d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:16.799045 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec10a5-e24e-4d42-af6c-800e7293681c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:16.799712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec10a5-e24e-4d42-af6c-800e7293681c admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:16.799996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec10a5-e24e-4d42-af6c-800e7293681c admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:16.800269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec10a5-e24e-4d42-af6c-800e7293681c admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:16.800999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec10a5-e24e-4d42-af6c-800e7293681c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:16.802622 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec10a5-e24e-4d42-af6c-800e7293681c admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:16.813839 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 992/1967] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:16.814682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec10a5-e24e-4d42-af6c-800e7293681c admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:16.816523 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-0cec10a5-e24e-4d42-af6c-800e7293681c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:16.829462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec10a5-e24e-4d42-af6c-800e7293681c admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:16.831959 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] The token cache is being invalidate because project 65ab32d45feb4576afc7be0fb6fece83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:16.832315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:16.832890 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:16.838316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 976/1968] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:16 2022] DELETE /identity/v3/projects/65ab32d45feb4576afc7be0fb6fece83 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:16.838316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:16.840574 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:16.844966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:16.845663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:16.845946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:16.846227 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] PATH_INFO: `/v3/projects/a203d4f4533049dd9dff836964badbf7` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:16.847479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a203d4f4533049dd9dff836964badbf7)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:16.849527 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:16.855613 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8d42a-2170-47db-88b6-b0822de1604d admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:16.862046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e73d8d-10eb-490f-8281-e855e148995d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:16.862785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e73d8d-10eb-490f-8281-e855e148995d admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:16.863066 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e73d8d-10eb-490f-8281-e855e148995d admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:16.863604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e73d8d-10eb-490f-8281-e855e148995d admin admin] PATH_INFO: `/v3/projects/4ea9dc61b45c433c9910dc767ac22873/users/88a3757553a64337a95c74636c0aab8e/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:16.865809 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e73d8d-10eb-490f-8281-e855e148995d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea9dc61b45c433c9910dc767ac22873, user_id=88a3757553a64337a95c74636c0aab8e, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:16.869656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e73d8d-10eb-490f-8281-e855e148995d admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:16.881316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 977/1969] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:16 2022] PUT /identity/v3/projects/4ea9dc61b45c433c9910dc767ac22873/users/88a3757553a64337a95c74636c0aab8e/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:16.882864 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fb7f1-695d-43de-8977-bc3312f3c23d None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:16.883309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fb7f1-695d-43de-8977-bc3312f3c23d None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:16.883796 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fb7f1-695d-43de-8977-bc3312f3c23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:16.909001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-634fb7f1-695d-43de-8977-bc3312f3c23d None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:16.910217 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-634fb7f1-695d-43de-8977-bc3312f3c23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:16.986197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 978/1970] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:16.986197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436d907-f558-4df3-a67e-612044064784 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:16.986197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436d907-f558-4df3-a67e-612044064784 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:16.986197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436d907-f558-4df3-a67e-612044064784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:16.989516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a203d4f4533049dd9dff836964badbf7', 'request_id': 'req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:16.996094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a203d4f4533049dd9dff836964badbf7', 'request_id': 'req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:17.017876 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-0436d907-f558-4df3-a67e-612044064784 None None] MFA Rules not processed for user `88a3757553a64337a95c74636c0aab8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:17.019146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0436d907-f558-4df3-a67e-612044064784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:17.101702 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 979/1971] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:16 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:17.102997 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc416ad-4f1d-4396-b8d2-bedd92b7b50b None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.103281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc416ad-4f1d-4396-b8d2-bedd92b7b50b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.103581 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc416ad-4f1d-4396-b8d2-bedd92b7b50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.106699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a203d4f4533049dd9dff836964badbf7', 'request_id': 'req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:17.120516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] The token cache is being invalidate because project a203d4f4533049dd9dff836964badbf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:17.120808 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:17.121504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:17.123535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 993/1972] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:16 2022] DELETE /identity/v3/projects/a203d4f4533049dd9dff836964badbf7 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:17.124453 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:17.125805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:17.127379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-fbc416ad-4f1d-4396-b8d2-bedd92b7b50b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:17.128680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-fbc416ad-4f1d-4396-b8d2-bedd92b7b50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:17.146679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:17.186017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6ce82-9e8f-41fe-9b59-ab5aeb58db5d admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:17.189347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 980/1973] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:17.189821 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48be18b-0be6-4def-8f25-2a62e8a57bd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:17.196784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea189c55-dba3-41c6-b0ed-6018b72cc648 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.198308 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48be18b-0be6-4def-8f25-2a62e8a57bd6 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.198834 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea189c55-dba3-41c6-b0ed-6018b72cc648 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.198962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48be18b-0be6-4def-8f25-2a62e8a57bd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.199087 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea189c55-dba3-41c6-b0ed-6018b72cc648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.199216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48be18b-0be6-4def-8f25-2a62e8a57bd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.199216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e48be18b-0be6-4def-8f25-2a62e8a57bd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:17.212197 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48be18b-0be6-4def-8f25-2a62e8a57bd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:17.214129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48be18b-0be6-4def-8f25-2a62e8a57bd6 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:17.216144 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ea189c55-dba3-41c6-b0ed-6018b72cc648 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:17.217370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ea189c55-dba3-41c6-b0ed-6018b72cc648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:17.224535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 994/1974] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:17 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:17.227415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 981/1975] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:17.242402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48be18b-0be6-4def-8f25-2a62e8a57bd6 service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:17.242402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48be18b-0be6-4def-8f25-2a62e8a57bd6 service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:17.242959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907cdd6-4809-4e4a-b3ad-ba8c7fdfc8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:17.244104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907cdd6-4809-4e4a-b3ad-ba8c7fdfc8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.244104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907cdd6-4809-4e4a-b3ad-ba8c7fdfc8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.244291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907cdd6-4809-4e4a-b3ad-ba8c7fdfc8c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.245003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8763ba3-8b36-4952-abfc-820810ceb0d9 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.245300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8763ba3-8b36-4952-abfc-820810ceb0d9 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.245583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8763ba3-8b36-4952-abfc-820810ceb0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.251598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907cdd6-4809-4e4a-b3ad-ba8c7fdfc8c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:17.252094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907cdd6-4809-4e4a-b3ad-ba8c7fdfc8c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:17.253849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907cdd6-4809-4e4a-b3ad-ba8c7fdfc8c7 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:17.266476 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-e8763ba3-8b36-4952-abfc-820810ceb0d9 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:17.267564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 995/1976] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:06:17.277099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e8763ba3-8b36-4952-abfc-820810ceb0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:17.277099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 982/1977] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:17.280290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907cdd6-4809-4e4a-b3ad-ba8c7fdfc8c7 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:17.281587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8763ba3-8b36-4952-abfc-820810ceb0d9 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:17.291328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907cdd6-4809-4e4a-b3ad-ba8c7fdfc8c7 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:17.291622 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8763ba3-8b36-4952-abfc-820810ceb0d9 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:17.295958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a5bbf-abdf-4e6d-9be9-a511c6da52a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:17.296238 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2fc8f-4d98-47d5-9fd9-cd18fac5481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:17.296574 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a5bbf-abdf-4e6d-9be9-a511c6da52a2 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.296875 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a5bbf-abdf-4e6d-9be9-a511c6da52a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.296976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2fc8f-4d98-47d5-9fd9-cd18fac5481c admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.297153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a5bbf-abdf-4e6d-9be9-a511c6da52a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.297275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2fc8f-4d98-47d5-9fd9-cd18fac5481c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.297515 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2fc8f-4d98-47d5-9fd9-cd18fac5481c admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.297939 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a5bbf-abdf-4e6d-9be9-a511c6da52a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:17.298125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2fc8f-4d98-47d5-9fd9-cd18fac5481c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:17.299075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2fc8f-4d98-47d5-9fd9-cd18fac5481c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:17.299892 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a5bbf-abdf-4e6d-9be9-a511c6da52a2 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:17.300709 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2fc8f-4d98-47d5-9fd9-cd18fac5481c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:17.315304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 983/1978] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:06:17.320622 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2fc8f-4d98-47d5-9fd9-cd18fac5481c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:17.330858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2fc8f-4d98-47d5-9fd9-cd18fac5481c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:17.335997 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ad847-8895-4265-acfa-05f37878553b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:17.336609 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ad847-8895-4265-acfa-05f37878553b admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.336894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ad847-8895-4265-acfa-05f37878553b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.337174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ad847-8895-4265-acfa-05f37878553b admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.337846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ad847-8895-4265-acfa-05f37878553b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:17.339840 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ad847-8895-4265-acfa-05f37878553b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:17.432896 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 996/1979] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:17 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:17.439224 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a5bbf-abdf-4e6d-9be9-a511c6da52a2 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:17.473511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 984/1980] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:17 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:06:17.477814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ad847-8895-4265-acfa-05f37878553b admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:17.510114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a5bbf-abdf-4e6d-9be9-a511c6da52a2 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:17.510865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:17.548068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ad847-8895-4265-acfa-05f37878553b admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:17.552841 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e160f-7dd0-45af-adbe-9f6fa8a0c9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:17.553460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e160f-7dd0-45af-adbe-9f6fa8a0c9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.553800 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e160f-7dd0-45af-adbe-9f6fa8a0c9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.554084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e160f-7dd0-45af-adbe-9f6fa8a0c9d7 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.554791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e160f-7dd0-45af-adbe-9f6fa8a0c9d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:17.556758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e160f-7dd0-45af-adbe-9f6fa8a0c9d7 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:17.578492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4960ebb-553d-4ce4-9975-20cf4e8c1d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:17.579123 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4960ebb-553d-4ce4-9975-20cf4e8c1d61 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.579409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4960ebb-553d-4ce4-9975-20cf4e8c1d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.579717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4960ebb-553d-4ce4-9975-20cf4e8c1d61 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.580435 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4960ebb-553d-4ce4-9975-20cf4e8c1d61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:17.582424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4960ebb-553d-4ce4-9975-20cf4e8c1d61 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:17.659301 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 985/1981] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:17 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:17.660278 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e160f-7dd0-45af-adbe-9f6fa8a0c9d7 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:17.720058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e160f-7dd0-45af-adbe-9f6fa8a0c9d7 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:17.724075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2a54c-a12c-42f6-8c99-81e63b386bd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:17.724750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2a54c-a12c-42f6-8c99-81e63b386bd3 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.725033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2a54c-a12c-42f6-8c99-81e63b386bd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.725314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2a54c-a12c-42f6-8c99-81e63b386bd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.727369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1ff2a54c-a12c-42f6-8c99-81e63b386bd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:17.787239 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 997/1982] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:17 2022] POST /identity/v3/users => generated 291 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:06:17.787239 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4960ebb-553d-4ce4-9975-20cf4e8c1d61 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:17.796024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2a54c-a12c-42f6-8c99-81e63b386bd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:17.798467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2a54c-a12c-42f6-8c99-81e63b386bd3 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:17.798626 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4960ebb-553d-4ce4-9975-20cf4e8c1d61 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:17.803157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15794f-1b02-4892-80d2-40dea0aec9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:17.804827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15794f-1b02-4892-80d2-40dea0aec9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.805250 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15794f-1b02-4892-80d2-40dea0aec9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.805633 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15794f-1b02-4892-80d2-40dea0aec9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.812319 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15794f-1b02-4892-80d2-40dea0aec9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:17.812319 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15794f-1b02-4892-80d2-40dea0aec9f3 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:17.812590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 986/1983] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:17 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:17.812590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2a54c-a12c-42f6-8c99-81e63b386bd3 service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:17.821913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 998/1984] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:17.822762 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2a54c-a12c-42f6-8c99-81e63b386bd3 service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:17.825999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15794f-1b02-4892-80d2-40dea0aec9f3 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:17.829897 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d2ff5-7202-4b46-b437-e3d03cade940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:17.830539 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d2ff5-7202-4b46-b437-e3d03cade940 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.830896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d2ff5-7202-4b46-b437-e3d03cade940 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.831184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d2ff5-7202-4b46-b437-e3d03cade940 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.831947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d2ff5-7202-4b46-b437-e3d03cade940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:17.833625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d2ff5-7202-4b46-b437-e3d03cade940 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:17.838888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15794f-1b02-4892-80d2-40dea0aec9f3 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:17.844635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61053484-5e4e-4271-b97c-7b6093ba6591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:17.845421 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61053484-5e4e-4271-b97c-7b6093ba6591 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.845639 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61053484-5e4e-4271-b97c-7b6093ba6591 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.846291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61053484-5e4e-4271-b97c-7b6093ba6591 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.847046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61053484-5e4e-4271-b97c-7b6093ba6591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:17.852252 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 987/1985] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:17.852423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61053484-5e4e-4271-b97c-7b6093ba6591 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:17.853465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d2ff5-7202-4b46-b437-e3d03cade940 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:17.873682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 999/1986] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:06:17.873910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d2ff5-7202-4b46-b437-e3d03cade940 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:17.873910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf535d0b-03ac-40bb-ba11-930a3339f5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:17.873910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf535d0b-03ac-40bb-ba11-930a3339f5ab admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.873910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf535d0b-03ac-40bb-ba11-930a3339f5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.874274 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf535d0b-03ac-40bb-ba11-930a3339f5ab admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.874274 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf535d0b-03ac-40bb-ba11-930a3339f5ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:17.875053 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61053484-5e4e-4271-b97c-7b6093ba6591 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:17.876189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf535d0b-03ac-40bb-ba11-930a3339f5ab admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:17.886067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61053484-5e4e-4271-b97c-7b6093ba6591 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:17.887210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 988/1987] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:17.894263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf535d0b-03ac-40bb-ba11-930a3339f5ab admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:17.894390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d134a-4326-4ef5-8c42-5e4ffa6b9440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:17.894390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d134a-4326-4ef5-8c42-5e4ffa6b9440 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.894943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d134a-4326-4ef5-8c42-5e4ffa6b9440 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.894943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d134a-4326-4ef5-8c42-5e4ffa6b9440 admin admin] PATH_INFO: `/v3/projects/4272758d01274f419bc75c85140daa06/users/2bf28a5c061c42668fc54a0c50af64ee/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.903143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf535d0b-03ac-40bb-ba11-930a3339f5ab admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:17.912765 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d134a-4326-4ef5-8c42-5e4ffa6b9440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4272758d01274f419bc75c85140daa06, user_id=2bf28a5c061c42668fc54a0c50af64ee, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:17.916961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d134a-4326-4ef5-8c42-5e4ffa6b9440 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:17.919364 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbae3c-d7d3-4e59-9ab1-0a397a19b32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:17.920163 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbae3c-d7d3-4e59-9ab1-0a397a19b32c admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:17.920565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbae3c-d7d3-4e59-9ab1-0a397a19b32c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:17.920911 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbae3c-d7d3-4e59-9ab1-0a397a19b32c admin admin] PATH_INFO: `/v3/projects/f082cde704ae4b658ce19bd1eaf169c4/users/58f8bddb70a947308794848293ba244c/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:17.937949 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbae3c-d7d3-4e59-9ab1-0a397a19b32c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f082cde704ae4b658ce19bd1eaf169c4, user_id=58f8bddb70a947308794848293ba244c, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:17.940948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbae3c-d7d3-4e59-9ab1-0a397a19b32c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:18.032917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:18.037613 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1000/1988] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:17 2022] PUT /identity/v3/projects/4272758d01274f419bc75c85140daa06/users/2bf28a5c061c42668fc54a0c50af64ee/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:18.043507 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 989/1989] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:17 2022] PUT /identity/v3/projects/f082cde704ae4b658ce19bd1eaf169c4/users/58f8bddb70a947308794848293ba244c/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:18.045940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784a22f-4665-4089-96e0-3b455a1edb84 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:18.046269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784a22f-4665-4089-96e0-3b455a1edb84 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:18.046609 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784a22f-4665-4089-96e0-3b455a1edb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:18.051093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df332c8-8e5b-4364-bc2b-81a5f11c64f1 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:18.051862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df332c8-8e5b-4364-bc2b-81a5f11c64f1 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:18.052554 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df332c8-8e5b-4364-bc2b-81a5f11c64f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:18.069039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c784a22f-4665-4089-96e0-3b455a1edb84 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:18.071166 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c784a22f-4665-4089-96e0-3b455a1edb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:18.087284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-2df332c8-8e5b-4364-bc2b-81a5f11c64f1 None None] MFA Rules not processed for user `58f8bddb70a947308794848293ba244c`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:18.088529 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-2df332c8-8e5b-4364-bc2b-81a5f11c64f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:18.139203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1001/1990] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:18.149668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784a22f-4665-4089-96e0-3b455a1edb84 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:18.156345 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c784a22f-4665-4089-96e0-3b455a1edb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:18.161313 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 990/1991] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:18.172879 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df332c8-8e5b-4364-bc2b-81a5f11c64f1 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:18.185271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df332c8-8e5b-4364-bc2b-81a5f11c64f1 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:18.203166 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85419cd6-a0cb-41cb-85a9-e7652393446f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:18.203969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85419cd6-a0cb-41cb-85a9-e7652393446f admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:18.204414 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85419cd6-a0cb-41cb-85a9-e7652393446f admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:18.216071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85419cd6-a0cb-41cb-85a9-e7652393446f admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:18.216614 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85419cd6-a0cb-41cb-85a9-e7652393446f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:18.218620 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85419cd6-a0cb-41cb-85a9-e7652393446f admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:18.243771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784a22f-4665-4089-96e0-3b455a1edb84 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:18.255216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7313c3-fd21-40ef-b760-efc8d2a4fc91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:18.257049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7313c3-fd21-40ef-b760-efc8d2a4fc91 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:18.257458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7313c3-fd21-40ef-b760-efc8d2a4fc91 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:18.257940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7313c3-fd21-40ef-b760-efc8d2a4fc91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:18.276800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7313c3-fd21-40ef-b760-efc8d2a4fc91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:18.278500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7313c3-fd21-40ef-b760-efc8d2a4fc91 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:18.290376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1002/1992] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:06:18.339853 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:18.387436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 991/1993] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:18 2022] POST /identity/v3/projects => generated 379 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:18.390411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7313c3-fd21-40ef-b760-efc8d2a4fc91 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:18.407634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7313c3-fd21-40ef-b760-efc8d2a4fc91 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:18.489453 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86588e17-edbc-42d3-8326-bd6e214ffb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:18.490800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86588e17-edbc-42d3-8326-bd6e214ffb7b admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:18.491413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86588e17-edbc-42d3-8326-bd6e214ffb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:18.491909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86588e17-edbc-42d3-8326-bd6e214ffb7b admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:18.497511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86588e17-edbc-42d3-8326-bd6e214ffb7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:18.497511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86588e17-edbc-42d3-8326-bd6e214ffb7b admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:18.650131 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1003/1994] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:18 2022] POST /identity/v3/users => generated 296 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:18.654434 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85419cd6-a0cb-41cb-85a9-e7652393446f admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:18.665401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85419cd6-a0cb-41cb-85a9-e7652393446f admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:18.670299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22978d16-92c0-4a65-a22a-e7ff96fed787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:18.670920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22978d16-92c0-4a65-a22a-e7ff96fed787 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:18.671203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22978d16-92c0-4a65-a22a-e7ff96fed787 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:18.671539 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22978d16-92c0-4a65-a22a-e7ff96fed787 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:18.672277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22978d16-92c0-4a65-a22a-e7ff96fed787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:18.673862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22978d16-92c0-4a65-a22a-e7ff96fed787 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:18.692060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 992/1995] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:18.695942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86588e17-edbc-42d3-8326-bd6e214ffb7b admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:18.707011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86588e17-edbc-42d3-8326-bd6e214ffb7b admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:18.712254 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937d63c-e9d7-4ee3-be4d-b17ead3aa825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:18.712940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937d63c-e9d7-4ee3-be4d-b17ead3aa825 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:18.713223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937d63c-e9d7-4ee3-be4d-b17ead3aa825 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:18.713496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937d63c-e9d7-4ee3-be4d-b17ead3aa825 admin admin] PATH_INFO: `/v3/projects/d51ebe04a391400fad2f09cdd9060e8e/users/bc2c0a8722ac4571b886f1035c4e235a/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:18.729754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937d63c-e9d7-4ee3-be4d-b17ead3aa825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d51ebe04a391400fad2f09cdd9060e8e, user_id=bc2c0a8722ac4571b886f1035c4e235a, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:18.735380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937d63c-e9d7-4ee3-be4d-b17ead3aa825 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:18.915614 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1004/1996] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:18 2022] PUT /identity/v3/projects/d51ebe04a391400fad2f09cdd9060e8e/users/bc2c0a8722ac4571b886f1035c4e235a/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:18.919598 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22978d16-92c0-4a65-a22a-e7ff96fed787 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:18.934833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22978d16-92c0-4a65-a22a-e7ff96fed787 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:19.004930 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e8b77-a727-42a9-b70f-f44b7091e782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:19.005563 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e8b77-a727-42a9-b70f-f44b7091e782 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:19.005843 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e8b77-a727-42a9-b70f-f44b7091e782 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:19.006117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e8b77-a727-42a9-b70f-f44b7091e782 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:19.006818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e8b77-a727-42a9-b70f-f44b7091e782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:19.009080 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e8b77-a727-42a9-b70f-f44b7091e782 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:19.020679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 993/1997] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:06:19.033161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937d63c-e9d7-4ee3-be4d-b17ead3aa825 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:19.036204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937d63c-e9d7-4ee3-be4d-b17ead3aa825 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:19.041831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0625b8-fba1-4a7c-a745-74e76d50d582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:19.042504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0625b8-fba1-4a7c-a745-74e76d50d582 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:19.042779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0625b8-fba1-4a7c-a745-74e76d50d582 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:19.043059 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0625b8-fba1-4a7c-a745-74e76d50d582 admin admin] PATH_INFO: `/v3/projects/d51ebe04a391400fad2f09cdd9060e8e/users/bc2c0a8722ac4571b886f1035c4e235a/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:19.045170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0625b8-fba1-4a7c-a745-74e76d50d582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d51ebe04a391400fad2f09cdd9060e8e, user_id=bc2c0a8722ac4571b886f1035c4e235a, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:19.047970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0625b8-fba1-4a7c-a745-74e76d50d582 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:19.071887 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1005/1998] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:19 2022] PUT /identity/v3/projects/d51ebe04a391400fad2f09cdd9060e8e/users/bc2c0a8722ac4571b886f1035c4e235a/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:19.076983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75dc58-0144-4301-b4d9-d94861a96baf None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:19.077495 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75dc58-0144-4301-b4d9-d94861a96baf None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:19.077587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75dc58-0144-4301-b4d9-d94861a96baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:19.107818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ec75dc58-0144-4301-b4d9-d94861a96baf None None] MFA Rules not processed for user `bc2c0a8722ac4571b886f1035c4e235a`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:19.109131 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ec75dc58-0144-4301-b4d9-d94861a96baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:19.164456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:19.179941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 994/1999] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:19.192214 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0625b8-fba1-4a7c-a745-74e76d50d582 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:19.276960 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0625b8-fba1-4a7c-a745-74e76d50d582 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:19.280840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d431f-5367-4cfd-9424-97262d4e4dc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:19.281504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d431f-5367-4cfd-9424-97262d4e4dc9 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:19.281743 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d431f-5367-4cfd-9424-97262d4e4dc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:19.282017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d431f-5367-4cfd-9424-97262d4e4dc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:19.292690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d431f-5367-4cfd-9424-97262d4e4dc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:19.293793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d431f-5367-4cfd-9424-97262d4e4dc9 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:19.300477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:19.307029 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1006/2000] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:19.918264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab542ec-d133-46d0-a5dd-545d14b940a4 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:19.920069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab542ec-d133-46d0-a5dd-545d14b940a4 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:19.920069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab542ec-d133-46d0-a5dd-545d14b940a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:19.955531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-5ab542ec-d133-46d0-a5dd-545d14b940a4 None None] MFA Rules not processed for user `2bf28a5c061c42668fc54a0c50af64ee`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:19.956759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-5ab542ec-d133-46d0-a5dd-545d14b940a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:20.025894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 995/2001] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:20.335855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:20.529968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:20.536595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d431f-5367-4cfd-9424-97262d4e4dc9 service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:20.547819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d431f-5367-4cfd-9424-97262d4e4dc9 service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:20.552083 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf63ce2-7c9b-42a4-9666-7318b4e47791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:20.552774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf63ce2-7c9b-42a4-9666-7318b4e47791 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:20.553119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf63ce2-7c9b-42a4-9666-7318b4e47791 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:20.554402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf63ce2-7c9b-42a4-9666-7318b4e47791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:20.563710 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf63ce2-7c9b-42a4-9666-7318b4e47791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:20.565238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf63ce2-7c9b-42a4-9666-7318b4e47791 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:20.574916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1007/2002] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:20.772564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab542ec-d133-46d0-a5dd-545d14b940a4 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:20.785788 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab542ec-d133-46d0-a5dd-545d14b940a4 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:20.793028 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c52f5-e2dd-4468-b893-5e955cba5eca tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2c0a8722ac4571b886f1035c4e235a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd51ebe04a391400fad2f09cdd9060e8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:20.794339 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c52f5-e2dd-4468-b893-5e955cba5eca tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:20.794634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c52f5-e2dd-4468-b893-5e955cba5eca tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:20.794917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c52f5-e2dd-4468-b893-5e955cba5eca tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:20.795994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029c52f5-e2dd-4468-b893-5e955cba5eca tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:20.796308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029c52f5-e2dd-4468-b893-5e955cba5eca tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:20.798284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029c52f5-e2dd-4468-b893-5e955cba5eca tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:20.817670 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 996/2003] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:06:20.819020 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf63ce2-7c9b-42a4-9666-7318b4e47791 service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:20.832251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf63ce2-7c9b-42a4-9666-7318b4e47791 service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:20.838063 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb86c5-8936-49a2-9be1-8d28247a21cb tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2c0a8722ac4571b886f1035c4e235a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd51ebe04a391400fad2f09cdd9060e8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:20.838733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb86c5-8936-49a2-9be1-8d28247a21cb tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:20.838987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb86c5-8936-49a2-9be1-8d28247a21cb tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:20.839266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb86c5-8936-49a2-9be1-8d28247a21cb tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:20.840229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb86c5-8936-49a2-9be1-8d28247a21cb tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:20.842411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb86c5-8936-49a2-9be1-8d28247a21cb tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:20.940995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1008/2004] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:20 2022] POST /identity/v3/projects => generated 354 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:20.943265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c52f5-e2dd-4468-b893-5e955cba5eca tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:20.953454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb86c5-8936-49a2-9be1-8d28247a21cb tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:20.955205 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-92eb86c5-8936-49a2-9be1-8d28247a21cb tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:20.955941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c52f5-e2dd-4468-b893-5e955cba5eca tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:21.021435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430039d-4273-4cda-92ba-fa09faf6875a tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2c0a8722ac4571b886f1035c4e235a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd51ebe04a391400fad2f09cdd9060e8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:21.022068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430039d-4273-4cda-92ba-fa09faf6875a tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:21.022863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430039d-4273-4cda-92ba-fa09faf6875a tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:21.023258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430039d-4273-4cda-92ba-fa09faf6875a tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:21.024280 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430039d-4273-4cda-92ba-fa09faf6875a tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:21.024671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430039d-4273-4cda-92ba-fa09faf6875a tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:21.026710 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430039d-4273-4cda-92ba-fa09faf6875a tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:21.028412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb86c5-8936-49a2-9be1-8d28247a21cb tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:21.033449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9d54b-2af3-4a6e-bf2e-6d78539ae841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:21.034100 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9d54b-2af3-4a6e-bf2e-6d78539ae841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:21.034385 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9d54b-2af3-4a6e-bf2e-6d78539ae841 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:21.038963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9d54b-2af3-4a6e-bf2e-6d78539ae841 admin admin] PATH_INFO: `/v3/users/82d2a0b56efd4a6a8fb77d1c18480637` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:21.038963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9d54b-2af3-4a6e-bf2e-6d78539ae841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82d2a0b56efd4a6a8fb77d1c18480637)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:21.038963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9d54b-2af3-4a6e-bf2e-6d78539ae841 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:21.043137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 997/2005] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:06:21.083319 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430039d-4273-4cda-92ba-fa09faf6875a tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:21.094813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430039d-4273-4cda-92ba-fa09faf6875a tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:21.099765 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8049c-24df-409b-9cb7-611f2aa05106 tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2c0a8722ac4571b886f1035c4e235a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd51ebe04a391400fad2f09cdd9060e8e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:21.101041 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8049c-24df-409b-9cb7-611f2aa05106 tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:21.101730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8049c-24df-409b-9cb7-611f2aa05106 tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:21.102213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8049c-24df-409b-9cb7-611f2aa05106 tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] PATH_INFO: `/v3/projects/8fc3f4d666c048828c4b11bf61e3efbd` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:21.111061 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8049c-24df-409b-9cb7-611f2aa05106 tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8fc3f4d666c048828c4b11bf61e3efbd)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:21.111061 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8049c-24df-409b-9cb7-611f2aa05106 tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:21.299278 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-19f8049c-24df-409b-9cb7-611f2aa05106 tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fc3f4d666c048828c4b11bf61e3efbd', 'request_id': 'req-19f8049c-24df-409b-9cb7-611f2aa05106'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:21.306068 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-19f8049c-24df-409b-9cb7-611f2aa05106 tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fc3f4d666c048828c4b11bf61e3efbd', 'request_id': 'req-19f8049c-24df-409b-9cb7-611f2aa05106'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:21.306561 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-80e9d54b-2af3-4a6e-bf2e-6d78539ae841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82d2a0b56efd4a6a8fb77d1c18480637'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:21.312593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-19f8049c-24df-409b-9cb7-611f2aa05106 tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fc3f4d666c048828c4b11bf61e3efbd', 'request_id': 'req-19f8049c-24df-409b-9cb7-611f2aa05106'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:21.318099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-80e9d54b-2af3-4a6e-bf2e-6d78539ae841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82d2a0b56efd4a6a8fb77d1c18480637'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:21.479726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-19f8049c-24df-409b-9cb7-611f2aa05106 tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] The token cache is being invalidate because project 8fc3f4d666c048828c4b11bf61e3efbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:21.479726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-19f8049c-24df-409b-9cb7-611f2aa05106 tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:21.479726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-19f8049c-24df-409b-9cb7-611f2aa05106 tempest-VolumeQuotaClassesTest-224272802 tempest-VolumeQuotaClassesTest-224272802-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:21.480369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 998/2006] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:21 2022] DELETE /identity/v3/projects/8fc3f4d666c048828c4b11bf61e3efbd => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:21.567929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-80e9d54b-2af3-4a6e-bf2e-6d78539ae841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82d2a0b56efd4a6a8fb77d1c18480637'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:21.584185 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-80e9d54b-2af3-4a6e-bf2e-6d78539ae841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82d2a0b56efd4a6a8fb77d1c18480637'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:21.590013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-80e9d54b-2af3-4a6e-bf2e-6d78539ae841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82d2a0b56efd4a6a8fb77d1c18480637'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:21.592584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1009/2007] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:20 2022] DELETE /identity/v3/users/82d2a0b56efd4a6a8fb77d1c18480637 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:21.597420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152addf-67d5-431d-9264-f31a8bddf493 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:21.597903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152addf-67d5-431d-9264-f31a8bddf493 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:21.598109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152addf-67d5-431d-9264-f31a8bddf493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:21.618900 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-1152addf-67d5-431d-9264-f31a8bddf493 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:21.620179 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1152addf-67d5-431d-9264-f31a8bddf493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:21.687895 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 999/2008] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:21.694645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9d54b-2af3-4a6e-bf2e-6d78539ae841 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:21.696268 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-80e9d54b-2af3-4a6e-bf2e-6d78539ae841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:21.760856 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9d54b-2af3-4a6e-bf2e-6d78539ae841 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:21.764757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24d5dc-db20-4c42-9ba1-1e534557f011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:21.765383 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24d5dc-db20-4c42-9ba1-1e534557f011 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:21.765666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24d5dc-db20-4c42-9ba1-1e534557f011 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:21.766078 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24d5dc-db20-4c42-9ba1-1e534557f011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:21.777832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24d5dc-db20-4c42-9ba1-1e534557f011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:21.779389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24d5dc-db20-4c42-9ba1-1e534557f011 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:21.803233 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1010/2009] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:22.145325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:22.277005 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:22.432318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152addf-67d5-431d-9264-f31a8bddf493 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:22.434098 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1152addf-67d5-431d-9264-f31a8bddf493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:22.453097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152addf-67d5-431d-9264-f31a8bddf493 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:22.461806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbeaea5-4381-4251-91da-d06a20fa1c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:22.463435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbeaea5-4381-4251-91da-d06a20fa1c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:22.463435 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbeaea5-4381-4251-91da-d06a20fa1c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:22.463952 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbeaea5-4381-4251-91da-d06a20fa1c2f admin admin] PATH_INFO: `/v3/users/58f8bddb70a947308794848293ba244c` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:22.466142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbeaea5-4381-4251-91da-d06a20fa1c2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=58f8bddb70a947308794848293ba244c)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:22.468329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbeaea5-4381-4251-91da-d06a20fa1c2f admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:22.516680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dec5d-b4f6-46c8-bcbc-54e11a3536ba None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:22.516988 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dec5d-b4f6-46c8-bcbc-54e11a3536ba None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:22.517264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dec5d-b4f6-46c8-bcbc-54e11a3536ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:22.538072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-d01dec5d-b4f6-46c8-bcbc-54e11a3536ba None None] MFA Rules not processed for user `2bf28a5c061c42668fc54a0c50af64ee`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:22.540210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d01dec5d-b4f6-46c8-bcbc-54e11a3536ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:22.637582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1011/2010] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:22.639106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dec5d-b4f6-46c8-bcbc-54e11a3536ba None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:22.641495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d01dec5d-b4f6-46c8-bcbc-54e11a3536ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:22.665186 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dec5d-b4f6-46c8-bcbc-54e11a3536ba None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:22.670343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:22.671005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:22.671289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:22.671618 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] PATH_INFO: `/v3/projects/0afc32c1f6ed4a88aac08facdbf7489f` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:22.672866 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0afc32c1f6ed4a88aac08facdbf7489f)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:22.674795 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:22.748786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3bbeaea5-4381-4251-91da-d06a20fa1c2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58f8bddb70a947308794848293ba244c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:22.749465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3bbeaea5-4381-4251-91da-d06a20fa1c2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58f8bddb70a947308794848293ba244c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:22.750078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3bbeaea5-4381-4251-91da-d06a20fa1c2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58f8bddb70a947308794848293ba244c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:22.759519 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3bbeaea5-4381-4251-91da-d06a20fa1c2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58f8bddb70a947308794848293ba244c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:22.774866 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3bbeaea5-4381-4251-91da-d06a20fa1c2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58f8bddb70a947308794848293ba244c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:22.823507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0afc32c1f6ed4a88aac08facdbf7489f', 'request_id': 'req-36327837-113d-4373-ba51-81c6306ea412'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:22.834499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0afc32c1f6ed4a88aac08facdbf7489f', 'request_id': 'req-36327837-113d-4373-ba51-81c6306ea412'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:22.894947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1000/2011] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:22 2022] DELETE /identity/v3/users/58f8bddb70a947308794848293ba244c => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:22.899944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb55d1-5e66-4bee-b981-5ce78f14abd9 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:22.900238 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb55d1-5e66-4bee-b981-5ce78f14abd9 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:22.900552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb55d1-5e66-4bee-b981-5ce78f14abd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:22.921437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-7fdb55d1-5e66-4bee-b981-5ce78f14abd9 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:22.922962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-7fdb55d1-5e66-4bee-b981-5ce78f14abd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:22.951959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0afc32c1f6ed4a88aac08facdbf7489f', 'request_id': 'req-36327837-113d-4373-ba51-81c6306ea412'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:22.965536 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] The token cache is being invalidate because project 0afc32c1f6ed4a88aac08facdbf7489f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:22.965836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:22.966746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:22.968425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1012/2012] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:22 2022] DELETE /identity/v3/projects/0afc32c1f6ed4a88aac08facdbf7489f => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:22.972566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:22.973986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:22.994364 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1001/2013] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:23.010217 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb55d1-5e66-4bee-b981-5ce78f14abd9 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:23.011759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-7fdb55d1-5e66-4bee-b981-5ce78f14abd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:23.063171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36327837-113d-4373-ba51-81c6306ea412 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:23.068836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42eeca-98be-46a3-bc75-91d2596f41df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:23.070276 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42eeca-98be-46a3-bc75-91d2596f41df admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:23.070737 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42eeca-98be-46a3-bc75-91d2596f41df admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:23.071220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42eeca-98be-46a3-bc75-91d2596f41df admin admin] PATH_INFO: `/v3/users/e2e6e7c7dcba47468962695d804a46bc` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:23.073059 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42eeca-98be-46a3-bc75-91d2596f41df admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2e6e7c7dcba47468962695d804a46bc)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:23.076090 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42eeca-98be-46a3-bc75-91d2596f41df admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:23.086368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb55d1-5e66-4bee-b981-5ce78f14abd9 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:23.091023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd0564-fbcb-4c80-a5af-24b5199cebfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:23.092013 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd0564-fbcb-4c80-a5af-24b5199cebfb service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:23.092455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd0564-fbcb-4c80-a5af-24b5199cebfb service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:23.092919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd0564-fbcb-4c80-a5af-24b5199cebfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:23.106847 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd0564-fbcb-4c80-a5af-24b5199cebfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:23.109108 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd0564-fbcb-4c80-a5af-24b5199cebfb service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:23.122469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1002/2014] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:06:23.298132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8e42eeca-98be-46a3-bc75-91d2596f41df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2e6e7c7dcba47468962695d804a46bc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:23.309691 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8e42eeca-98be-46a3-bc75-91d2596f41df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2e6e7c7dcba47468962695d804a46bc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:23.413843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8e42eeca-98be-46a3-bc75-91d2596f41df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2e6e7c7dcba47468962695d804a46bc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:23.431371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8e42eeca-98be-46a3-bc75-91d2596f41df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2e6e7c7dcba47468962695d804a46bc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:23.432717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8e42eeca-98be-46a3-bc75-91d2596f41df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2e6e7c7dcba47468962695d804a46bc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:23.439818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1013/2015] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:22 2022] DELETE /identity/v3/users/e2e6e7c7dcba47468962695d804a46bc => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:23.636240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:23.761007 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd0564-fbcb-4c80-a5af-24b5199cebfb service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:23.767919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e8fd0564-fbcb-4c80-a5af-24b5199cebfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:23.784978 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd0564-fbcb-4c80-a5af-24b5199cebfb service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:23.872524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:23.873540 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:23.873540 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:23.873735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] PATH_INFO: `/v3/projects/f082cde704ae4b658ce19bd1eaf169c4` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:23.874919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f082cde704ae4b658ce19bd1eaf169c4)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:23.876804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:24.028293 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42eeca-98be-46a3-bc75-91d2596f41df admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:24.029680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-8e42eeca-98be-46a3-bc75-91d2596f41df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:24.035259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:24.048060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f082cde704ae4b658ce19bd1eaf169c4', 'request_id': 'req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.057675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f082cde704ae4b658ce19bd1eaf169c4', 'request_id': 'req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.062889 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42eeca-98be-46a3-bc75-91d2596f41df admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:24.063957 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f082cde704ae4b658ce19bd1eaf169c4', 'request_id': 'req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.122452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:24.123112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:24.123392 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:24.123705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] PATH_INFO: `/v3/projects/06bfdd1e85394a89a138b62bb99bb9f5` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:24.124878 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06bfdd1e85394a89a138b62bb99bb9f5)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:24.126781 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:24.148392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] The token cache is being invalidate because project f082cde704ae4b658ce19bd1eaf169c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:24.148966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.150028 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.152495 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1003/2016] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:23 2022] DELETE /identity/v3/projects/f082cde704ae4b658ce19bd1eaf169c4 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:24.156784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:24.158340 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:24.174088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b01704-0e5a-418e-bf45-8c9e27a8a1d8 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:24.180291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ceb48-943b-47f5-94ff-30b461bcbc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:24.180966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ceb48-943b-47f5-94ff-30b461bcbc6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:24.181248 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ceb48-943b-47f5-94ff-30b461bcbc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:24.181529 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ceb48-943b-47f5-94ff-30b461bcbc6f admin admin] PATH_INFO: `/v3/users/bc2c0a8722ac4571b886f1035c4e235a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:24.182915 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ceb48-943b-47f5-94ff-30b461bcbc6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc2c0a8722ac4571b886f1035c4e235a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:24.185138 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ceb48-943b-47f5-94ff-30b461bcbc6f admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:24.292128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06bfdd1e85394a89a138b62bb99bb9f5', 'request_id': 'req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.298348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06bfdd1e85394a89a138b62bb99bb9f5', 'request_id': 'req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.414652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06bfdd1e85394a89a138b62bb99bb9f5', 'request_id': 'req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.437167 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] The token cache is being invalidate because project 06bfdd1e85394a89a138b62bb99bb9f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:24.437733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.438992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.441856 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1014/2017] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:24 2022] DELETE /identity/v3/projects/06bfdd1e85394a89a138b62bb99bb9f5 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:24.449217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:24.450217 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a85ceb48-943b-47f5-94ff-30b461bcbc6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc2c0a8722ac4571b886f1035c4e235a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.450522 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a85ceb48-943b-47f5-94ff-30b461bcbc6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc2c0a8722ac4571b886f1035c4e235a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.451140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a85ceb48-943b-47f5-94ff-30b461bcbc6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc2c0a8722ac4571b886f1035c4e235a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.451385 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:24.459935 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a85ceb48-943b-47f5-94ff-30b461bcbc6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc2c0a8722ac4571b886f1035c4e235a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.485354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a85ceb48-943b-47f5-94ff-30b461bcbc6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc2c0a8722ac4571b886f1035c4e235a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.526747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50b0d5-8b02-4a4d-b39d-f65840ce2ac1 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:24.532227 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d83ee-34f4-4aec-91dc-2e063c699508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:24.532891 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d83ee-34f4-4aec-91dc-2e063c699508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:24.533164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d83ee-34f4-4aec-91dc-2e063c699508 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:24.533446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d83ee-34f4-4aec-91dc-2e063c699508 admin admin] PATH_INFO: `/v3/users/fa6ed4f8ba084ead9ffd3f09da3dafbe` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:24.534725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d83ee-34f4-4aec-91dc-2e063c699508 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa6ed4f8ba084ead9ffd3f09da3dafbe)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:24.536748 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d83ee-34f4-4aec-91dc-2e063c699508 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:24.567441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1004/2018] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:24 2022] DELETE /identity/v3/users/bc2c0a8722ac4571b886f1035c4e235a => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:24.749226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-675d83ee-34f4-4aec-91dc-2e063c699508 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa6ed4f8ba084ead9ffd3f09da3dafbe'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.757900 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-675d83ee-34f4-4aec-91dc-2e063c699508 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa6ed4f8ba084ead9ffd3f09da3dafbe'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.853652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-675d83ee-34f4-4aec-91dc-2e063c699508 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa6ed4f8ba084ead9ffd3f09da3dafbe'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.874252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-675d83ee-34f4-4aec-91dc-2e063c699508 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa6ed4f8ba084ead9ffd3f09da3dafbe'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.876536 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-675d83ee-34f4-4aec-91dc-2e063c699508 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa6ed4f8ba084ead9ffd3f09da3dafbe'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:24.886252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1015/2019] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:24 2022] DELETE /identity/v3/users/fa6ed4f8ba084ead9ffd3f09da3dafbe => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:25.108542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ceb48-943b-47f5-94ff-30b461bcbc6f admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:25.110358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a85ceb48-943b-47f5-94ff-30b461bcbc6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:25.114547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:25.211425 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ceb48-943b-47f5-94ff-30b461bcbc6f admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:25.275627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:25.276921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:25.276921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:25.276921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9 admin admin] PATH_INFO: `/v3/projects/d51ebe04a391400fad2f09cdd9060e8e` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:25.278192 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d51ebe04a391400fad2f09cdd9060e8e)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:25.280246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:25.376578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d83ee-34f4-4aec-91dc-2e063c699508 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:25.378593 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-675d83ee-34f4-4aec-91dc-2e063c699508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:25.394281 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d83ee-34f4-4aec-91dc-2e063c699508 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:25.399981 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c04071-0f76-4d70-a18e-aab00324c605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:25.408241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c04071-0f76-4d70-a18e-aab00324c605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:25.408628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c04071-0f76-4d70-a18e-aab00324c605 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:25.409193 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c04071-0f76-4d70-a18e-aab00324c605 admin admin] PATH_INFO: `/v3/projects/4ce09d7520e9436e83b97433a464729b` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:25.410285 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c04071-0f76-4d70-a18e-aab00324c605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ce09d7520e9436e83b97433a464729b)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:25.412264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c04071-0f76-4d70-a18e-aab00324c605 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:25.439684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd51ebe04a391400fad2f09cdd9060e8e', 'request_id': 'req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:25.448353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd51ebe04a391400fad2f09cdd9060e8e', 'request_id': 'req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:25.454791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd51ebe04a391400fad2f09cdd9060e8e', 'request_id': 'req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:25.574585 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-49c04071-0f76-4d70-a18e-aab00324c605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ce09d7520e9436e83b97433a464729b', 'request_id': 'req-49c04071-0f76-4d70-a18e-aab00324c605'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:25.582236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-49c04071-0f76-4d70-a18e-aab00324c605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ce09d7520e9436e83b97433a464729b', 'request_id': 'req-49c04071-0f76-4d70-a18e-aab00324c605'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:25.582459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9 admin admin] The token cache is being invalidate because project d51ebe04a391400fad2f09cdd9060e8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:25.582760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:25.583500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-777d3af7-3c1c-41ad-adc4-178ed4c80bd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:25.585545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1005/2020] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:25 2022] DELETE /identity/v3/projects/d51ebe04a391400fad2f09cdd9060e8e => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:25.590272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4b15f-715a-47e4-b471-b2ac7c38d733 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:25.590560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4b15f-715a-47e4-b471-b2ac7c38d733 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:25.590840 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4b15f-715a-47e4-b471-b2ac7c38d733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:25.617356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-1ba4b15f-715a-47e4-b471-b2ac7c38d733 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:25.618599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1ba4b15f-715a-47e4-b471-b2ac7c38d733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:25.689936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1006/2021] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:25.696319 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-49c04071-0f76-4d70-a18e-aab00324c605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ce09d7520e9436e83b97433a464729b', 'request_id': 'req-49c04071-0f76-4d70-a18e-aab00324c605'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:25.698892 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a9bbf-d2c4-473e-b62e-70020a92a3d2 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:25.699177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a9bbf-d2c4-473e-b62e-70020a92a3d2 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:25.699493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a9bbf-d2c4-473e-b62e-70020a92a3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:25.716343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-49c04071-0f76-4d70-a18e-aab00324c605 admin admin] The token cache is being invalidate because project 4ce09d7520e9436e83b97433a464729b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:25.717026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-49c04071-0f76-4d70-a18e-aab00324c605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:25.718220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-49c04071-0f76-4d70-a18e-aab00324c605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:25.720289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-091a9bbf-d2c4-473e-b62e-70020a92a3d2 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:25.721120 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1016/2022] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:25 2022] DELETE /identity/v3/projects/4ce09d7520e9436e83b97433a464729b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:25.724628 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-091a9bbf-d2c4-473e-b62e-70020a92a3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:25.729071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dd49c-adf8-4a57-a632-f366c70045f4 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:25.729495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dd49c-adf8-4a57-a632-f366c70045f4 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:25.729955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dd49c-adf8-4a57-a632-f366c70045f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:25.748594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1007/2023] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:25.752660 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a9bbf-d2c4-473e-b62e-70020a92a3d2 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:25.765320 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a9bbf-d2c4-473e-b62e-70020a92a3d2 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:25.773643 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-938dd49c-adf8-4a57-a632-f366c70045f4 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:25.775221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-938dd49c-adf8-4a57-a632-f366c70045f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:25.781464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eee4b-6fb6-46c8-b62c-9717065906f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:25.782122 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eee4b-6fb6-46c8-b62c-9717065906f1 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:25.782411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eee4b-6fb6-46c8-b62c-9717065906f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:25.782685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eee4b-6fb6-46c8-b62c-9717065906f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:25.783423 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735eee4b-6fb6-46c8-b62c-9717065906f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:25.783845 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735eee4b-6fb6-46c8-b62c-9717065906f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:25.785518 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735eee4b-6fb6-46c8-b62c-9717065906f1 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:25.789211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:25.803632 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1017/2025] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:25.804288 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1008/2024] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:06:25.812293 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb11d51-7325-4b40-86dd-289211a446ce None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:25.812957 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb11d51-7325-4b40-86dd-289211a446ce None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:25.813616 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb11d51-7325-4b40-86dd-289211a446ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:25.817448 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eee4b-6fb6-46c8-b62c-9717065906f1 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:25.835490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eee4b-6fb6-46c8-b62c-9717065906f1 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:25.838402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-edb11d51-7325-4b40-86dd-289211a446ce None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:25.839748 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-edb11d51-7325-4b40-86dd-289211a446ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:25.841990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af21417-883b-4205-90b1-72af6c0fc612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:25.842723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af21417-883b-4205-90b1-72af6c0fc612 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:25.843014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af21417-883b-4205-90b1-72af6c0fc612 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:25.843292 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af21417-883b-4205-90b1-72af6c0fc612 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:25.844752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af21417-883b-4205-90b1-72af6c0fc612 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:25.846321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af21417-883b-4205-90b1-72af6c0fc612 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:25.851510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1018/2026] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:25.856109 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb11d51-7325-4b40-86dd-289211a446ce None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:25.867667 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:25.868858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb11d51-7325-4b40-86dd-289211a446ce None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:25.874297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c00d6-ac1d-4bb0-99dc-b6e70802a747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:25.874937 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c00d6-ac1d-4bb0-99dc-b6e70802a747 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:25.875209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c00d6-ac1d-4bb0-99dc-b6e70802a747 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:25.875499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c00d6-ac1d-4bb0-99dc-b6e70802a747 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:25.876268 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c00d6-ac1d-4bb0-99dc-b6e70802a747 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:25.876574 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c00d6-ac1d-4bb0-99dc-b6e70802a747 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:25.878234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c00d6-ac1d-4bb0-99dc-b6e70802a747 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:25.892885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1019/2027] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:06:25.896511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c00d6-ac1d-4bb0-99dc-b6e70802a747 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:25.907543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c00d6-ac1d-4bb0-99dc-b6e70802a747 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:25.912353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2f0f1-8826-442b-a85d-97fa1d42b7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:25.913091 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2f0f1-8826-442b-a85d-97fa1d42b7cd admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:25.913371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2f0f1-8826-442b-a85d-97fa1d42b7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:25.913641 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2f0f1-8826-442b-a85d-97fa1d42b7cd admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:25.914503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2f0f1-8826-442b-a85d-97fa1d42b7cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:25.916521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2f0f1-8826-442b-a85d-97fa1d42b7cd admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:25.971201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1009/2028] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:25 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:25.975603 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af21417-883b-4205-90b1-72af6c0fc612 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:26.042675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af21417-883b-4205-90b1-72af6c0fc612 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:26.044624 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1020/2029] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:25 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:06:26.050244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2f0f1-8826-442b-a85d-97fa1d42b7cd admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:26.113282 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd445f33-09db-4528-a764-d49a87cded17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:26.114242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd445f33-09db-4528-a764-d49a87cded17 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:26.114242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd445f33-09db-4528-a764-d49a87cded17 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:26.114525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd445f33-09db-4528-a764-d49a87cded17 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:26.115224 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd445f33-09db-4528-a764-d49a87cded17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:26.118167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd445f33-09db-4528-a764-d49a87cded17 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:26.122097 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2f0f1-8826-442b-a85d-97fa1d42b7cd admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:26.127200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540751ab-8506-45d7-83bd-0bdfbdb79e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:26.127881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540751ab-8506-45d7-83bd-0bdfbdb79e43 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:26.128179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540751ab-8506-45d7-83bd-0bdfbdb79e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:26.128450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540751ab-8506-45d7-83bd-0bdfbdb79e43 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:26.129131 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540751ab-8506-45d7-83bd-0bdfbdb79e43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:26.131351 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540751ab-8506-45d7-83bd-0bdfbdb79e43 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:26.319861 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1010/2030] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:25 2022] POST /identity/v3/users => generated 289 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:06:26.323894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1021/2031] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:26 2022] POST /identity/v3/users => generated 283 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:06:26.324852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd445f33-09db-4528-a764-d49a87cded17 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:26.324971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540751ab-8506-45d7-83bd-0bdfbdb79e43 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:26.336094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:26.343751 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd445f33-09db-4528-a764-d49a87cded17 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:26.343978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540751ab-8506-45d7-83bd-0bdfbdb79e43 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:26.344082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5f639-3c1b-4a27-9dff-d332085024ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:26.344082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5f639-3c1b-4a27-9dff-d332085024ca admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:26.344245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5f639-3c1b-4a27-9dff-d332085024ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:26.344523 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5f639-3c1b-4a27-9dff-d332085024ca admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:26.344713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee59312-94b7-425a-bc07-5ba6b639347d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:26.345231 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d5f639-3c1b-4a27-9dff-d332085024ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:26.345364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee59312-94b7-425a-bc07-5ba6b639347d admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:26.345600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee59312-94b7-425a-bc07-5ba6b639347d admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:26.345869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee59312-94b7-425a-bc07-5ba6b639347d admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:26.346564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee59312-94b7-425a-bc07-5ba6b639347d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:26.346899 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d5f639-3c1b-4a27-9dff-d332085024ca admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:26.348303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee59312-94b7-425a-bc07-5ba6b639347d admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:26.358743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1011/2032] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:26.361104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1022/2033] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:26.365498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee59312-94b7-425a-bc07-5ba6b639347d admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:26.365732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5f639-3c1b-4a27-9dff-d332085024ca admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:26.381605 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5f639-3c1b-4a27-9dff-d332085024ca admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:26.383404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee59312-94b7-425a-bc07-5ba6b639347d admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:26.387705 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e5bf9-e043-4e0f-8616-03b604dae6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:26.388437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e5bf9-e043-4e0f-8616-03b604dae6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:26.388721 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e5bf9-e043-4e0f-8616-03b604dae6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:26.389002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e5bf9-e043-4e0f-8616-03b604dae6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:26.389716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e5bf9-e043-4e0f-8616-03b604dae6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:26.389892 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61980da-7c69-4b9f-b7a0-00ea3313423a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:26.390726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61980da-7c69-4b9f-b7a0-00ea3313423a admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:26.391122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61980da-7c69-4b9f-b7a0-00ea3313423a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:26.391450 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e5bf9-e043-4e0f-8616-03b604dae6d8 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:26.391644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61980da-7c69-4b9f-b7a0-00ea3313423a admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:26.392498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61980da-7c69-4b9f-b7a0-00ea3313423a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:26.394280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61980da-7c69-4b9f-b7a0-00ea3313423a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:26.406830 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1012/2034] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:26.408328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1023/2035] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:06:26.412402 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e5bf9-e043-4e0f-8616-03b604dae6d8 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:26.413298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61980da-7c69-4b9f-b7a0-00ea3313423a admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:26.424576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e5bf9-e043-4e0f-8616-03b604dae6d8 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:26.429677 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61980da-7c69-4b9f-b7a0-00ea3313423a admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:26.430439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee3305-fa5f-4111-bcf0-3b5c6655bc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:26.431186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee3305-fa5f-4111-bcf0-3b5c6655bc1b admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:26.432130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee3305-fa5f-4111-bcf0-3b5c6655bc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:26.432130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee3305-fa5f-4111-bcf0-3b5c6655bc1b admin admin] PATH_INFO: `/v3/projects/c9cfcb9e78de43ae827f021047253214/users/d03d43d573ae4f3d8dbecc99ef6824b0/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:26.438508 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e337fd-18d5-4bb2-9205-8ec9ce0d37d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:26.439292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e337fd-18d5-4bb2-9205-8ec9ce0d37d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:26.439665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e337fd-18d5-4bb2-9205-8ec9ce0d37d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:26.440005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e337fd-18d5-4bb2-9205-8ec9ce0d37d3 admin admin] PATH_INFO: `/v3/projects/68941b1b130b48b087fcac2cd59af3e7/users/84ff87abb45648b0b75e4af76ff11c56/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:26.458782 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee3305-fa5f-4111-bcf0-3b5c6655bc1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9cfcb9e78de43ae827f021047253214, user_id=d03d43d573ae4f3d8dbecc99ef6824b0, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:26.461332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e337fd-18d5-4bb2-9205-8ec9ce0d37d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68941b1b130b48b087fcac2cd59af3e7, user_id=84ff87abb45648b0b75e4af76ff11c56, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:26.463712 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee3305-fa5f-4111-bcf0-3b5c6655bc1b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:26.466479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e337fd-18d5-4bb2-9205-8ec9ce0d37d3 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:26.548532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1013/2036] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:26 2022] PUT /identity/v3/projects/c9cfcb9e78de43ae827f021047253214/users/d03d43d573ae4f3d8dbecc99ef6824b0/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:26.555920 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1024/2037] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:26 2022] PUT /identity/v3/projects/68941b1b130b48b087fcac2cd59af3e7/users/84ff87abb45648b0b75e4af76ff11c56/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:26.565243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ac3cf-48b7-4eb4-a13e-83668cbfb662 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:26.565624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ac3cf-48b7-4eb4-a13e-83668cbfb662 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:26.567228 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb9b15-ba50-48c4-be2e-c4a6be1631d0 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:26.567766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb9b15-ba50-48c4-be2e-c4a6be1631d0 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:26.568033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb9b15-ba50-48c4-be2e-c4a6be1631d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:26.568897 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ac3cf-48b7-4eb4-a13e-83668cbfb662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:26.610901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-4ebb9b15-ba50-48c4-be2e-c4a6be1631d0 None None] MFA Rules not processed for user `d03d43d573ae4f3d8dbecc99ef6824b0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:26.611783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-255ac3cf-48b7-4eb4-a13e-83668cbfb662 None None] MFA Rules not processed for user `84ff87abb45648b0b75e4af76ff11c56`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:26.613108 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-255ac3cf-48b7-4eb4-a13e-83668cbfb662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:26.613568 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4ebb9b15-ba50-48c4-be2e-c4a6be1631d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:26.731284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1014/2038] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:26.736509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ac3cf-48b7-4eb4-a13e-83668cbfb662 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:26.752871 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1025/2039] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:26.767750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb9b15-ba50-48c4-be2e-c4a6be1631d0 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:26.769558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4ebb9b15-ba50-48c4-be2e-c4a6be1631d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:26.831382 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:26.840415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ac3cf-48b7-4eb4-a13e-83668cbfb662 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:26.853993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41029a70-3948-4069-8888-59945a4a5e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:26.854706 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41029a70-3948-4069-8888-59945a4a5e87 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:26.855581 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41029a70-3948-4069-8888-59945a4a5e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:26.856035 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41029a70-3948-4069-8888-59945a4a5e87 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:26.856829 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41029a70-3948-4069-8888-59945a4a5e87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:26.864379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41029a70-3948-4069-8888-59945a4a5e87 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:26.897729 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb9b15-ba50-48c4-be2e-c4a6be1631d0 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:26.902102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0fe1d-83bf-4aea-be25-d507fa6d88c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b153935910c142aa80792e111872318a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:26.902832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0fe1d-83bf-4aea-be25-d507fa6d88c9 service glance] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:26.903127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0fe1d-83bf-4aea-be25-d507fa6d88c9 service glance] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:26.903389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0fe1d-83bf-4aea-be25-d507fa6d88c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:26.914615 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0fe1d-83bf-4aea-be25-d507fa6d88c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:26.916224 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0fe1d-83bf-4aea-be25-d507fa6d88c9 service glance] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:26.926283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1026/2040] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:06:26.985321 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:27.012900 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1015/2041] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:26 2022] POST /identity/v3/projects => generated 367 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:06:27.013254 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0fe1d-83bf-4aea-be25-d507fa6d88c9 service glance] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:27.025762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0fe1d-83bf-4aea-be25-d507fa6d88c9 service glance] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:27.114215 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b794ec-93e6-4a4a-8b68-c0a76cfa92a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:27.115249 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b794ec-93e6-4a4a-8b68-c0a76cfa92a2 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:27.115818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b794ec-93e6-4a4a-8b68-c0a76cfa92a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:27.116313 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b794ec-93e6-4a4a-8b68-c0a76cfa92a2 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:27.117569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b794ec-93e6-4a4a-8b68-c0a76cfa92a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:27.123390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b794ec-93e6-4a4a-8b68-c0a76cfa92a2 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:27.372494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1027/2042] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:27 2022] POST /identity/v3/users => generated 290 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:27.378362 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41029a70-3948-4069-8888-59945a4a5e87 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:27.391033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41029a70-3948-4069-8888-59945a4a5e87 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:27.396487 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11cf821-5408-42a5-97dd-ed26cce0d76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:27.398113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cf821-5408-42a5-97dd-ed26cce0d76b admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:27.398556 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cf821-5408-42a5-97dd-ed26cce0d76b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:27.398971 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cf821-5408-42a5-97dd-ed26cce0d76b admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:27.400264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11cf821-5408-42a5-97dd-ed26cce0d76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:27.402193 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11cf821-5408-42a5-97dd-ed26cce0d76b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:27.418716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1016/2043] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:27.420217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b794ec-93e6-4a4a-8b68-c0a76cfa92a2 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:27.432454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b794ec-93e6-4a4a-8b68-c0a76cfa92a2 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:27.437911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2cddd-a16a-4754-84d5-bd3a168dfd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:27.438588 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2cddd-a16a-4754-84d5-bd3a168dfd8b admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:27.440387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2cddd-a16a-4754-84d5-bd3a168dfd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:27.440387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2cddd-a16a-4754-84d5-bd3a168dfd8b admin admin] PATH_INFO: `/v3/projects/cd819e39f5a0466296d8057328681125/users/b00e432584474444a18e899ac09cd5e3/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:27.457308 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2cddd-a16a-4754-84d5-bd3a168dfd8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd819e39f5a0466296d8057328681125, user_id=b00e432584474444a18e899ac09cd5e3, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:27.460055 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2cddd-a16a-4754-84d5-bd3a168dfd8b admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:27.609017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1028/2044] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:27 2022] PUT /identity/v3/projects/cd819e39f5a0466296d8057328681125/users/b00e432584474444a18e899ac09cd5e3/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:27.613634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11cf821-5408-42a5-97dd-ed26cce0d76b admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:27.625651 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11cf821-5408-42a5-97dd-ed26cce0d76b admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:27.711693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125815b5-e174-4b84-b20f-e4088b8d8079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:27.712162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125815b5-e174-4b84-b20f-e4088b8d8079 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:27.712455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125815b5-e174-4b84-b20f-e4088b8d8079 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:27.712744 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125815b5-e174-4b84-b20f-e4088b8d8079 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:27.713473 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125815b5-e174-4b84-b20f-e4088b8d8079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:27.715092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125815b5-e174-4b84-b20f-e4088b8d8079 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:27.726865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1017/2045] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:27.733519 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2cddd-a16a-4754-84d5-bd3a168dfd8b admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:27.747651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2cddd-a16a-4754-84d5-bd3a168dfd8b admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:27.760363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ab0cf-ea78-4385-b3f3-4ce74e86f2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:27.761268 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ab0cf-ea78-4385-b3f3-4ce74e86f2b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:27.761658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ab0cf-ea78-4385-b3f3-4ce74e86f2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:27.762051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ab0cf-ea78-4385-b3f3-4ce74e86f2b7 admin admin] PATH_INFO: `/v3/projects/cd819e39f5a0466296d8057328681125/users/b00e432584474444a18e899ac09cd5e3/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:27.764462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ab0cf-ea78-4385-b3f3-4ce74e86f2b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd819e39f5a0466296d8057328681125, user_id=b00e432584474444a18e899ac09cd5e3, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:27.767407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ab0cf-ea78-4385-b3f3-4ce74e86f2b7 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:27.787528 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1029/2046] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:27 2022] PUT /identity/v3/projects/cd819e39f5a0466296d8057328681125/users/b00e432584474444a18e899ac09cd5e3/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:27.811621 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1e185-4025-489c-a1fd-6c54fffebed2 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:27.812289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1e185-4025-489c-a1fd-6c54fffebed2 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:27.812787 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1e185-4025-489c-a1fd-6c54fffebed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:27.851989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-eaa1e185-4025-489c-a1fd-6c54fffebed2 None None] MFA Rules not processed for user `b00e432584474444a18e899ac09cd5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:27.853537 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-eaa1e185-4025-489c-a1fd-6c54fffebed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:27.950360 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1018/2047] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:27.996352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ab0cf-ea78-4385-b3f3-4ce74e86f2b7 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:27.999243 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d85ab0cf-ea78-4385-b3f3-4ce74e86f2b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:28.151716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ab0cf-ea78-4385-b3f3-4ce74e86f2b7 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:28.156206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df1616-10fd-44ca-aaaa-204fbd3d9761 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:28.156948 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df1616-10fd-44ca-aaaa-204fbd3d9761 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:28.157332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df1616-10fd-44ca-aaaa-204fbd3d9761 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:28.157758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df1616-10fd-44ca-aaaa-204fbd3d9761 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:28.170322 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:28.172210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df1616-10fd-44ca-aaaa-204fbd3d9761 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:28.174266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df1616-10fd-44ca-aaaa-204fbd3d9761 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:28.196399 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1030/2048] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:27 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:28.689846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:28.749117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1e185-4025-489c-a1fd-6c54fffebed2 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:28.751149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-eaa1e185-4025-489c-a1fd-6c54fffebed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:28.772031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1e185-4025-489c-a1fd-6c54fffebed2 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:28.779344 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077093b1-bbc1-401e-9dac-09762beacb57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:28.780405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077093b1-bbc1-401e-9dac-09762beacb57 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:28.780848 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077093b1-bbc1-401e-9dac-09762beacb57 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:28.781312 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077093b1-bbc1-401e-9dac-09762beacb57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:28.856946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077093b1-bbc1-401e-9dac-09762beacb57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:28.858528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077093b1-bbc1-401e-9dac-09762beacb57 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:28.870902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1019/2049] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:30.316191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:30.640069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:31.843440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:31.998064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:32.564593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:32.785066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:34.079429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df1616-10fd-44ca-aaaa-204fbd3d9761 service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:34.080895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-13df1616-10fd-44ca-aaaa-204fbd3d9761 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:34.189492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df1616-10fd-44ca-aaaa-204fbd3d9761 service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:34.195573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca39749-19ed-4ce8-a917-9bfd6f2fd17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:34.196189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca39749-19ed-4ce8-a917-9bfd6f2fd17a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:34.196472 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca39749-19ed-4ce8-a917-9bfd6f2fd17a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:34.196912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca39749-19ed-4ce8-a917-9bfd6f2fd17a admin admin] PATH_INFO: `/v3/users/5288337071f74459acf4fb1d5abb7d07` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:34.198350 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca39749-19ed-4ce8-a917-9bfd6f2fd17a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5288337071f74459acf4fb1d5abb7d07)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:34.200406 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca39749-19ed-4ce8-a917-9bfd6f2fd17a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:34.439910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:34.653955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2ca39749-19ed-4ce8-a917-9bfd6f2fd17a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5288337071f74459acf4fb1d5abb7d07'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:34.661376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2ca39749-19ed-4ce8-a917-9bfd6f2fd17a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5288337071f74459acf4fb1d5abb7d07'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:34.786084 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2ca39749-19ed-4ce8-a917-9bfd6f2fd17a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5288337071f74459acf4fb1d5abb7d07'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:34.824258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2ca39749-19ed-4ce8-a917-9bfd6f2fd17a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5288337071f74459acf4fb1d5abb7d07'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:34.825230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-2ca39749-19ed-4ce8-a917-9bfd6f2fd17a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5288337071f74459acf4fb1d5abb7d07'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:34.837965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1031/2050] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:34 2022] DELETE /identity/v3/users/5288337071f74459acf4fb1d5abb7d07 => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:34.843200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c96c-74f4-4493-bfb7-ec2c37e7bd07 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:34.843940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c96c-74f4-4493-bfb7-ec2c37e7bd07 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:34.843940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c96c-74f4-4493-bfb7-ec2c37e7bd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:34.874743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-da44c96c-74f4-4493-bfb7-ec2c37e7bd07 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:34.876168 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-da44c96c-74f4-4493-bfb7-ec2c37e7bd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:34.942132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:34.964105 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1020/2051] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:34.993206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca39749-19ed-4ce8-a917-9bfd6f2fd17a admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:34.994704 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2ca39749-19ed-4ce8-a917-9bfd6f2fd17a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:35.108923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca39749-19ed-4ce8-a917-9bfd6f2fd17a admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:35.114273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a276e-35e2-409b-982a-34f768a74a9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:35.115299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a276e-35e2-409b-982a-34f768a74a9a service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:35.115831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a276e-35e2-409b-982a-34f768a74a9a service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:35.116286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a276e-35e2-409b-982a-34f768a74a9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:35.127628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a276e-35e2-409b-982a-34f768a74a9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:35.129502 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a276e-35e2-409b-982a-34f768a74a9a service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:35.141461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1032/2052] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:35.990920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44c96c-74f4-4493-bfb7-ec2c37e7bd07 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:35.993985 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-da44c96c-74f4-4493-bfb7-ec2c37e7bd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:36.021973 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44c96c-74f4-4493-bfb7-ec2c37e7bd07 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:36.031537 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:36.032240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:36.032536 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:36.032823 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] PATH_INFO: `/v3/projects/71732c42b01744839e51ba6bd3af2e16` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:36.034096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71732c42b01744839e51ba6bd3af2e16)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:36.036512 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:36.230865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71732c42b01744839e51ba6bd3af2e16', 'request_id': 'req-02b42f81-c6b3-4ecc-819d-5a0db37be817'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:36.238519 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71732c42b01744839e51ba6bd3af2e16', 'request_id': 'req-02b42f81-c6b3-4ecc-819d-5a0db37be817'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:36.250009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71732c42b01744839e51ba6bd3af2e16', 'request_id': 'req-02b42f81-c6b3-4ecc-819d-5a0db37be817'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:36.354110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] The token cache is being invalidate because project 71732c42b01744839e51ba6bd3af2e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:36.354421 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:36.356273 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:36.357945 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1021/2053] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:35 2022] DELETE /identity/v3/projects/71732c42b01744839e51ba6bd3af2e16 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:36.363288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a276e-35e2-409b-982a-34f768a74a9a service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:36.374781 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a276e-35e2-409b-982a-34f768a74a9a service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:36.470822 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5465f4-a458-4419-b0da-bc546919f937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:36.471727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5465f4-a458-4419-b0da-bc546919f937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:36.472037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5465f4-a458-4419-b0da-bc546919f937 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:36.472459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5465f4-a458-4419-b0da-bc546919f937 admin admin] PATH_INFO: `/v3/users/88a3757553a64337a95c74636c0aab8e` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:36.474257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5465f4-a458-4419-b0da-bc546919f937 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88a3757553a64337a95c74636c0aab8e)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:36.477629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5465f4-a458-4419-b0da-bc546919f937 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:36.838652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-0b5465f4-a458-4419-b0da-bc546919f937 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88a3757553a64337a95c74636c0aab8e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:36.840811 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:36.849919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-0b5465f4-a458-4419-b0da-bc546919f937 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88a3757553a64337a95c74636c0aab8e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:36.956666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-0b5465f4-a458-4419-b0da-bc546919f937 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88a3757553a64337a95c74636c0aab8e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:36.968376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:36.971241 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:36.982776 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-0b5465f4-a458-4419-b0da-bc546919f937 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88a3757553a64337a95c74636c0aab8e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:36.983701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-0b5465f4-a458-4419-b0da-bc546919f937 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88a3757553a64337a95c74636c0aab8e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:36.991514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1033/2054] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:36 2022] DELETE /identity/v3/users/88a3757553a64337a95c74636c0aab8e => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:36.995492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b42f81-c6b3-4ecc-819d-5a0db37be817 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:37.115201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a4c01-55e6-4fe0-98ba-02e50849ee58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:37.117540 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a4c01-55e6-4fe0-98ba-02e50849ee58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:37.118357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a4c01-55e6-4fe0-98ba-02e50849ee58 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:37.118705 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a4c01-55e6-4fe0-98ba-02e50849ee58 admin admin] PATH_INFO: `/v3/users/84ff87abb45648b0b75e4af76ff11c56` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:37.121932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a4c01-55e6-4fe0-98ba-02e50849ee58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84ff87abb45648b0b75e4af76ff11c56)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:37.123997 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a4c01-55e6-4fe0-98ba-02e50849ee58 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:37.479972 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-029a4c01-55e6-4fe0-98ba-02e50849ee58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84ff87abb45648b0b75e4af76ff11c56'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:37.480914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-029a4c01-55e6-4fe0-98ba-02e50849ee58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84ff87abb45648b0b75e4af76ff11c56'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:37.481717 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-029a4c01-55e6-4fe0-98ba-02e50849ee58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84ff87abb45648b0b75e4af76ff11c56'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:37.490118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-029a4c01-55e6-4fe0-98ba-02e50849ee58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84ff87abb45648b0b75e4af76ff11c56'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:37.507591 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-029a4c01-55e6-4fe0-98ba-02e50849ee58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84ff87abb45648b0b75e4af76ff11c56'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:37.645346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:37.654833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1022/2055] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:36 2022] DELETE /identity/v3/users/84ff87abb45648b0b75e4af76ff11c56 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:37.657056 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb2583-e901-4511-a305-96cd9bc66a18 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:37.658145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb2583-e901-4511-a305-96cd9bc66a18 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:37.658442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb2583-e901-4511-a305-96cd9bc66a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:37.687195 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-69fb2583-e901-4511-a305-96cd9bc66a18 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:37.688632 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-69fb2583-e901-4511-a305-96cd9bc66a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:37.772113 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1034/2056] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:37.783033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a4c01-55e6-4fe0-98ba-02e50849ee58 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:37.784893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-029a4c01-55e6-4fe0-98ba-02e50849ee58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:37.866035 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a4c01-55e6-4fe0-98ba-02e50849ee58 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:37.871063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2f2bf-59d3-47c8-9488-f311ca7599ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:37.872020 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2f2bf-59d3-47c8-9488-f311ca7599ef service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:37.872320 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2f2bf-59d3-47c8-9488-f311ca7599ef service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:37.872597 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2f2bf-59d3-47c8-9488-f311ca7599ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:37.885593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2f2bf-59d3-47c8-9488-f311ca7599ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:37.890364 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2f2bf-59d3-47c8-9488-f311ca7599ef service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:37.892061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb2583-e901-4511-a305-96cd9bc66a18 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:37.897380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-69fb2583-e901-4511-a305-96cd9bc66a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:37.903485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1023/2057] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:37.924888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb2583-e901-4511-a305-96cd9bc66a18 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:37.932494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:37.933911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:37.934423 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:37.934815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] PATH_INFO: `/v3/projects/4ea9dc61b45c433c9910dc767ac22873` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:37.936273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ea9dc61b45c433c9910dc767ac22873)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:37.938338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:38.005255 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee04caa-5adf-49ba-a9ba-cf10f64218cc None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:38.006551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee04caa-5adf-49ba-a9ba-cf10f64218cc None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:38.007128 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee04caa-5adf-49ba-a9ba-cf10f64218cc None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:38.008864 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1024/2058] 10.222.0.94 () {54 vars in 921 bytes} [Tue Jul 26 15:06:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 15:06:38.042051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0fa25-7395-40c9-8a38-9691be5ea557 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:38.042477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0fa25-7395-40c9-8a38-9691be5ea557 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:38.042823 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0fa25-7395-40c9-8a38-9691be5ea557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:38.067261 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-f5d0fa25-7395-40c9-8a38-9691be5ea557 None None] MFA Rules not processed for user `42a0728e2f6249bdb45d6f44fb15aa4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:38.069187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f5d0fa25-7395-40c9-8a38-9691be5ea557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:38.091383 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ea9dc61b45c433c9910dc767ac22873', 'request_id': 'req-68b2bcd4-351d-44e4-87d7-1b5febead947'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:38.100628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ea9dc61b45c433c9910dc767ac22873', 'request_id': 'req-68b2bcd4-351d-44e4-87d7-1b5febead947'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:38.152193 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1025/2059] 10.222.0.94 () {58 vars in 1018 bytes} [Tue Jul 26 15:06:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:38.172608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0fa25-7395-40c9-8a38-9691be5ea557 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:38.174182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f5d0fa25-7395-40c9-8a38-9691be5ea557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:38.181091 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ea9dc61b45c433c9910dc767ac22873', 'request_id': 'req-68b2bcd4-351d-44e4-87d7-1b5febead947'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:38.199103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] The token cache is being invalidate because project 4ea9dc61b45c433c9910dc767ac22873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:38.199608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:38.200492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:38.202711 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1035/2060] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:37 2022] DELETE /identity/v3/projects/4ea9dc61b45c433c9910dc767ac22873 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:38.204156 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:38.206608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:38.234213 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2bcd4-351d-44e4-87d7-1b5febead947 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:38.241486 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4124c-a53a-49be-b15f-0706dfb6f1e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a0728e2f6249bdb45d6f44fb15aa4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:38.242461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4124c-a53a-49be-b15f-0706dfb6f1e2 service placement] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:38.243508 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4124c-a53a-49be-b15f-0706dfb6f1e2 service placement] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:38.243978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4124c-a53a-49be-b15f-0706dfb6f1e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:38.248231 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a0c4124c-a53a-49be-b15f-0706dfb6f1e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:38.272544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4124c-a53a-49be-b15f-0706dfb6f1e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:38.275124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4124c-a53a-49be-b15f-0706dfb6f1e2 service placement] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:38.284506 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0fa25-7395-40c9-8a38-9691be5ea557 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:38.289507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1036/2061] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:38.291620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd5c38-d9b0-4759-befc-c35df3e6fec2 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:38.292216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd5c38-d9b0-4759-befc-c35df3e6fec2 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:38.296529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd5c38-d9b0-4759-befc-c35df3e6fec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:38.296865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603eaeeb-8107-4531-8e1f-3643ee9cde2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:38.296865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603eaeeb-8107-4531-8e1f-3643ee9cde2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:38.296865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603eaeeb-8107-4531-8e1f-3643ee9cde2d admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:38.296865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603eaeeb-8107-4531-8e1f-3643ee9cde2d admin admin] PATH_INFO: `/v3/users/c15d08b7eb014af3afc4d0a6e032f740` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:38.297165 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603eaeeb-8107-4531-8e1f-3643ee9cde2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c15d08b7eb014af3afc4d0a6e032f740)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:38.297973 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603eaeeb-8107-4531-8e1f-3643ee9cde2d admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:38.325209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-22dd5c38-d9b0-4759-befc-c35df3e6fec2 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:38.326520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-22dd5c38-d9b0-4759-befc-c35df3e6fec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:38.343505 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1037/2062] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:38.349684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a8cc9-9291-4c07-88fc-2b9e24f1444f None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:38.350057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a8cc9-9291-4c07-88fc-2b9e24f1444f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:38.350250 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a8cc9-9291-4c07-88fc-2b9e24f1444f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:38.371337 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b46a8cc9-9291-4c07-88fc-2b9e24f1444f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:38.372791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b46a8cc9-9291-4c07-88fc-2b9e24f1444f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:38.380812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1038/2063] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:38.385283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a8cc9-9291-4c07-88fc-2b9e24f1444f None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:38.397544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a8cc9-9291-4c07-88fc-2b9e24f1444f None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:38.402662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d629b-9847-43a1-9137-39a3bf999d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:38.403278 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d629b-9847-43a1-9137-39a3bf999d98 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:38.403579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d629b-9847-43a1-9137-39a3bf999d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:38.403855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d629b-9847-43a1-9137-39a3bf999d98 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:38.404553 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d629b-9847-43a1-9137-39a3bf999d98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:38.404858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d629b-9847-43a1-9137-39a3bf999d98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:38.406464 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d629b-9847-43a1-9137-39a3bf999d98 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:38.430294 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1039/2064] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:06:38.431761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d629b-9847-43a1-9137-39a3bf999d98 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:38.471270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d629b-9847-43a1-9137-39a3bf999d98 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:38.479245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aaf827-6112-4839-8af2-025e57e12e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:38.480266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaf827-6112-4839-8af2-025e57e12e41 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:38.480736 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaf827-6112-4839-8af2-025e57e12e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:38.481479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaf827-6112-4839-8af2-025e57e12e41 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:38.482623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aaf827-6112-4839-8af2-025e57e12e41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:38.486610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aaf827-6112-4839-8af2-025e57e12e41 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:38.507544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:38.539223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-603eaeeb-8107-4531-8e1f-3643ee9cde2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c15d08b7eb014af3afc4d0a6e032f740'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:38.539846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-603eaeeb-8107-4531-8e1f-3643ee9cde2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c15d08b7eb014af3afc4d0a6e032f740'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:38.540615 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-603eaeeb-8107-4531-8e1f-3643ee9cde2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c15d08b7eb014af3afc4d0a6e032f740'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:38.550550 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-603eaeeb-8107-4531-8e1f-3643ee9cde2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c15d08b7eb014af3afc4d0a6e032f740'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:38.567569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-603eaeeb-8107-4531-8e1f-3643ee9cde2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c15d08b7eb014af3afc4d0a6e032f740'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:38.685062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1040/2065] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:38 2022] POST /identity/v3/projects => generated 375 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:06:38.686050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aaf827-6112-4839-8af2-025e57e12e41 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:38.687823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-35aaf827-6112-4839-8af2-025e57e12e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:38.720618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1026/2066] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:38 2022] DELETE /identity/v3/users/c15d08b7eb014af3afc4d0a6e032f740 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:38.723358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603eaeeb-8107-4531-8e1f-3643ee9cde2d admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:38.725016 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-603eaeeb-8107-4531-8e1f-3643ee9cde2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:38.766334 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aaf827-6112-4839-8af2-025e57e12e41 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:38.801026 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603eaeeb-8107-4531-8e1f-3643ee9cde2d admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:38.808102 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7538215-43fc-40d4-afd7-bde5f8ab11d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:38.810719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7538215-43fc-40d4-afd7-bde5f8ab11d4 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:38.811174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7538215-43fc-40d4-afd7-bde5f8ab11d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:38.811453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7538215-43fc-40d4-afd7-bde5f8ab11d4 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:38.812232 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7538215-43fc-40d4-afd7-bde5f8ab11d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:38.814472 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7538215-43fc-40d4-afd7-bde5f8ab11d4 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:38.832644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592b1b7-592f-4089-9716-78fe56454904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:38.833297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592b1b7-592f-4089-9716-78fe56454904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:38.833657 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592b1b7-592f-4089-9716-78fe56454904 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:38.834071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592b1b7-592f-4089-9716-78fe56454904 admin admin] PATH_INFO: `/v3/projects/68941b1b130b48b087fcac2cd59af3e7` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:38.835166 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592b1b7-592f-4089-9716-78fe56454904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68941b1b130b48b087fcac2cd59af3e7)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:38.837190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592b1b7-592f-4089-9716-78fe56454904 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:38.900691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1027/2067] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:38 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:38.904607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7538215-43fc-40d4-afd7-bde5f8ab11d4 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:38.914565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7538215-43fc-40d4-afd7-bde5f8ab11d4 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:38.919394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41971f59-5c6b-4e7b-afbc-ffec40de10f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:38.920066 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41971f59-5c6b-4e7b-afbc-ffec40de10f0 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:38.920351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41971f59-5c6b-4e7b-afbc-ffec40de10f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:38.920626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41971f59-5c6b-4e7b-afbc-ffec40de10f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:38.921526 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41971f59-5c6b-4e7b-afbc-ffec40de10f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:38.923170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41971f59-5c6b-4e7b-afbc-ffec40de10f0 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:38.934305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1028/2068] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:38.938405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41971f59-5c6b-4e7b-afbc-ffec40de10f0 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:38.948846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41971f59-5c6b-4e7b-afbc-ffec40de10f0 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:38.953490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da878608-b391-40b3-ac11-e0f07b680443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:38.954090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da878608-b391-40b3-ac11-e0f07b680443 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:38.954385 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da878608-b391-40b3-ac11-e0f07b680443 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:38.954656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da878608-b391-40b3-ac11-e0f07b680443 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:38.955377 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da878608-b391-40b3-ac11-e0f07b680443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:38.957018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da878608-b391-40b3-ac11-e0f07b680443 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:38.969310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1029/2069] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:38.973519 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da878608-b391-40b3-ac11-e0f07b680443 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:38.984489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da878608-b391-40b3-ac11-e0f07b680443 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:38.994073 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd8883-af8c-44ff-b836-ab7f3731bc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:38.994809 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd8883-af8c-44ff-b836-ab7f3731bc24 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:38.995155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd8883-af8c-44ff-b836-ab7f3731bc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:38.995413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd8883-af8c-44ff-b836-ab7f3731bc24 admin admin] PATH_INFO: `/v3/projects/ab09a0cd208b4b2b93bd2c292b8f4e1f/users/0757c09591c2499fa07e17ba5a88f9d3/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:39.012875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd8883-af8c-44ff-b836-ab7f3731bc24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab09a0cd208b4b2b93bd2c292b8f4e1f, user_id=0757c09591c2499fa07e17ba5a88f9d3, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:39.014088 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a592b1b7-592f-4089-9716-78fe56454904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68941b1b130b48b087fcac2cd59af3e7', 'request_id': 'req-a592b1b7-592f-4089-9716-78fe56454904'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.016504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd8883-af8c-44ff-b836-ab7f3731bc24 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:39.030945 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a592b1b7-592f-4089-9716-78fe56454904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68941b1b130b48b087fcac2cd59af3e7', 'request_id': 'req-a592b1b7-592f-4089-9716-78fe56454904'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.121750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a592b1b7-592f-4089-9716-78fe56454904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68941b1b130b48b087fcac2cd59af3e7', 'request_id': 'req-a592b1b7-592f-4089-9716-78fe56454904'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.127993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1030/2070] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:38 2022] PUT /identity/v3/projects/ab09a0cd208b4b2b93bd2c292b8f4e1f/users/0757c09591c2499fa07e17ba5a88f9d3/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:39.128875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd8883-af8c-44ff-b836-ab7f3731bc24 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:39.130504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c8fd8883-af8c-44ff-b836-ab7f3731bc24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:39.135650 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a592b1b7-592f-4089-9716-78fe56454904 admin admin] The token cache is being invalidate because project 68941b1b130b48b087fcac2cd59af3e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:39.135947 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a592b1b7-592f-4089-9716-78fe56454904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.136895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a592b1b7-592f-4089-9716-78fe56454904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.139487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1041/2071] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:38 2022] DELETE /identity/v3/projects/68941b1b130b48b087fcac2cd59af3e7 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:39.144087 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790a5c2-361b-4c85-96b7-8dfdc4eddb2a None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:39.144364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790a5c2-361b-4c85-96b7-8dfdc4eddb2a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:39.144646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790a5c2-361b-4c85-96b7-8dfdc4eddb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:39.176217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-1790a5c2-361b-4c85-96b7-8dfdc4eddb2a None None] MFA Rules not processed for user `0757c09591c2499fa07e17ba5a88f9d3`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:39.177522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1790a5c2-361b-4c85-96b7-8dfdc4eddb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:39.213384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:39.224211 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd8883-af8c-44ff-b836-ab7f3731bc24 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:39.231425 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:39.232343 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:39.232693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:39.232974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] PATH_INFO: `/v3/projects/bb18a5955f814887bcfa9650bcaa2ce9` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:39.234210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb18a5955f814887bcfa9650bcaa2ce9)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:39.236239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:39.267373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1042/2072] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:39.267373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790a5c2-361b-4c85-96b7-8dfdc4eddb2a None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:39.267373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1790a5c2-361b-4c85-96b7-8dfdc4eddb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:39.282468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790a5c2-361b-4c85-96b7-8dfdc4eddb2a None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:39.288587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e2376-76f9-4bd8-93c4-e9fd94f893c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:39.289228 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e2376-76f9-4bd8-93c4-e9fd94f893c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:39.289508 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e2376-76f9-4bd8-93c4-e9fd94f893c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:39.289781 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e2376-76f9-4bd8-93c4-e9fd94f893c0 admin admin] PATH_INFO: `/v3/users/b00e432584474444a18e899ac09cd5e3` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:39.290985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e2376-76f9-4bd8-93c4-e9fd94f893c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b00e432584474444a18e899ac09cd5e3)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:39.293000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e2376-76f9-4bd8-93c4-e9fd94f893c0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:39.465845 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb18a5955f814887bcfa9650bcaa2ce9', 'request_id': 'req-ffeda867-fb14-4dfa-887e-ade8ba431fb5'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.472376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb18a5955f814887bcfa9650bcaa2ce9', 'request_id': 'req-ffeda867-fb14-4dfa-887e-ade8ba431fb5'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.478949 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb18a5955f814887bcfa9650bcaa2ce9', 'request_id': 'req-ffeda867-fb14-4dfa-887e-ade8ba431fb5'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.593554 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] The token cache is being invalidate because project bb18a5955f814887bcfa9650bcaa2ce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:39.594118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.594705 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.597201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1031/2073] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:39 2022] DELETE /identity/v3/projects/bb18a5955f814887bcfa9650bcaa2ce9 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:39.603313 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:39.605073 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:39.605839 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-ac5e2376-76f9-4bd8-93c4-e9fd94f893c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b00e432584474444a18e899ac09cd5e3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.614485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-ac5e2376-76f9-4bd8-93c4-e9fd94f893c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b00e432584474444a18e899ac09cd5e3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.707386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-ac5e2376-76f9-4bd8-93c4-e9fd94f893c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b00e432584474444a18e899ac09cd5e3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.711923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeda867-fb14-4dfa-887e-ade8ba431fb5 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:39.717281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b0c87-d222-4e66-aa36-14424fc8bd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:39.717823 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b0c87-d222-4e66-aa36-14424fc8bd27 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:39.718106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b0c87-d222-4e66-aa36-14424fc8bd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:39.718452 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b0c87-d222-4e66-aa36-14424fc8bd27 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:39.719141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b0c87-d222-4e66-aa36-14424fc8bd27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:39.721158 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b0c87-d222-4e66-aa36-14424fc8bd27 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:39.727767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-ac5e2376-76f9-4bd8-93c4-e9fd94f893c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b00e432584474444a18e899ac09cd5e3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.729649 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-ac5e2376-76f9-4bd8-93c4-e9fd94f893c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b00e432584474444a18e899ac09cd5e3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:39.743899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1043/2074] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:39 2022] DELETE /identity/v3/users/b00e432584474444a18e899ac09cd5e3 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:39.745454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3f35b-d57d-49e6-8743-754ce85ba0b4 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:39.745780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3f35b-d57d-49e6-8743-754ce85ba0b4 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:39.746054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3f35b-d57d-49e6-8743-754ce85ba0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:39.773191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-4ad3f35b-d57d-49e6-8743-754ce85ba0b4 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:39.775545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4ad3f35b-d57d-49e6-8743-754ce85ba0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:39.863652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1044/2075] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:39.865472 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdbd66-b525-4b2c-8e78-3da000de0e8c None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:39.865803 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdbd66-b525-4b2c-8e78-3da000de0e8c None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:39.866080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdbd66-b525-4b2c-8e78-3da000de0e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:39.879520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1032/2076] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:39 2022] POST /identity/v3/projects => generated 373 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:39.886786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b0c87-d222-4e66-aa36-14424fc8bd27 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:39.889171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a85b0c87-d222-4e66-aa36-14424fc8bd27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:39.897195 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-6ccdbd66-b525-4b2c-8e78-3da000de0e8c None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:39.901637 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:39.906241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6ccdbd66-b525-4b2c-8e78-3da000de0e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:39.983616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b0c87-d222-4e66-aa36-14424fc8bd27 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:39.988479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35323c-e0f2-4ac8-b193-650eff117e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:39.989106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35323c-e0f2-4ac8-b193-650eff117e5e admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:39.989382 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35323c-e0f2-4ac8-b193-650eff117e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:39.989802 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35323c-e0f2-4ac8-b193-650eff117e5e admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:39.990941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35323c-e0f2-4ac8-b193-650eff117e5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:39.993337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35323c-e0f2-4ac8-b193-650eff117e5e admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:40.003609 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1045/2077] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:40.004861 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdbd66-b525-4b2c-8e78-3da000de0e8c None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:40.014294 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdbd66-b525-4b2c-8e78-3da000de0e8c None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:40.019098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79beb9ec-2a99-4c26-bd78-a5a1e1a312a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:40.019757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beb9ec-2a99-4c26-bd78-a5a1e1a312a2 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.020042 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beb9ec-2a99-4c26-bd78-a5a1e1a312a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.020292 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:40.020785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beb9ec-2a99-4c26-bd78-a5a1e1a312a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.021039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beb9ec-2a99-4c26-bd78-a5a1e1a312a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:40.021344 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beb9ec-2a99-4c26-bd78-a5a1e1a312a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:40.022981 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beb9ec-2a99-4c26-bd78-a5a1e1a312a2 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:40.035218 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1046/2078] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:06:40.040276 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79beb9ec-2a99-4c26-bd78-a5a1e1a312a2 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:40.052452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79beb9ec-2a99-4c26-bd78-a5a1e1a312a2 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:40.060161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c606223-0388-4ed4-afcf-e1db915ec422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:40.061235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606223-0388-4ed4-afcf-e1db915ec422 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.061915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606223-0388-4ed4-afcf-e1db915ec422 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.062084 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606223-0388-4ed4-afcf-e1db915ec422 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.063034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c606223-0388-4ed4-afcf-e1db915ec422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:40.065471 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c606223-0388-4ed4-afcf-e1db915ec422 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:40.093835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1033/2079] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:39 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:40.098007 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35323c-e0f2-4ac8-b193-650eff117e5e admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:40.110226 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35323c-e0f2-4ac8-b193-650eff117e5e admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:40.117084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc89857-082b-4604-8180-9d576da6a01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:40.121111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc89857-082b-4604-8180-9d576da6a01a admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.121111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc89857-082b-4604-8180-9d576da6a01a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.121111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc89857-082b-4604-8180-9d576da6a01a admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.121111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc89857-082b-4604-8180-9d576da6a01a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:40.121111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc89857-082b-4604-8180-9d576da6a01a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:40.134857 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1034/2080] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:40.139338 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc89857-082b-4604-8180-9d576da6a01a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:40.151138 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc89857-082b-4604-8180-9d576da6a01a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:40.156227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4e450-245f-426a-b602-8e0f18f2ece9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:40.156912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e450-245f-426a-b602-8e0f18f2ece9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.157191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e450-245f-426a-b602-8e0f18f2ece9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.157468 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e450-245f-426a-b602-8e0f18f2ece9 admin admin] PATH_INFO: `/v3/projects/a9f8df8d077c496d996ea884ce3fb12a/users/d805e6f407a5432aa2c2fdf0c63f5256/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.174669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4e450-245f-426a-b602-8e0f18f2ece9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9f8df8d077c496d996ea884ce3fb12a, user_id=d805e6f407a5432aa2c2fdf0c63f5256, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:40.180970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1047/2081] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:40 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:06:40.181164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4e450-245f-426a-b602-8e0f18f2ece9 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:40.182451 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c606223-0388-4ed4-afcf-e1db915ec422 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:40.259382 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1035/2082] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:40 2022] PUT /identity/v3/projects/a9f8df8d077c496d996ea884ce3fb12a/users/d805e6f407a5432aa2c2fdf0c63f5256/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:40.274020 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c606223-0388-4ed4-afcf-e1db915ec422 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:40.280956 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4e450-245f-426a-b602-8e0f18f2ece9 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:40.303487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:40.350979 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94d7c1-b207-4623-a230-253878e80a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:40.351802 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94d7c1-b207-4623-a230-253878e80a56 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.352134 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94d7c1-b207-4623-a230-253878e80a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.352412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94d7c1-b207-4623-a230-253878e80a56 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.353154 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94d7c1-b207-4623-a230-253878e80a56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:40.357011 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:40.362153 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4e450-245f-426a-b602-8e0f18f2ece9 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:40.368703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df7b73-dd1d-4180-af4f-f4b1f645bc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:40.369427 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df7b73-dd1d-4180-af4f-f4b1f645bc6d admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.369716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df7b73-dd1d-4180-af4f-f4b1f645bc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.369996 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df7b73-dd1d-4180-af4f-f4b1f645bc6d admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.370103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94d7c1-b207-4623-a230-253878e80a56 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:40.371065 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df7b73-dd1d-4180-af4f-f4b1f645bc6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:40.373310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df7b73-dd1d-4180-af4f-f4b1f645bc6d admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:40.392641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1036/2083] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:06:40.393503 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df7b73-dd1d-4180-af4f-f4b1f645bc6d admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:40.395573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c2df7b73-dd1d-4180-af4f-f4b1f645bc6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:40.410545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df7b73-dd1d-4180-af4f-f4b1f645bc6d admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:40.415795 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c992d87-ed0b-4337-9a49-80b44727d04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:40.416469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c992d87-ed0b-4337-9a49-80b44727d04d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.416762 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c992d87-ed0b-4337-9a49-80b44727d04d admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.417045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c992d87-ed0b-4337-9a49-80b44727d04d admin admin] PATH_INFO: `/v3/projects/cd819e39f5a0466296d8057328681125` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.418372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c992d87-ed0b-4337-9a49-80b44727d04d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd819e39f5a0466296d8057328681125)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:40.420580 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c992d87-ed0b-4337-9a49-80b44727d04d admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:40.489421 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1048/2084] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:40 2022] POST /identity/v3/users => generated 284 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:06:40.489421 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94d7c1-b207-4623-a230-253878e80a56 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:40.497836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94d7c1-b207-4623-a230-253878e80a56 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:40.502545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba762c-0e86-4eae-bbc8-5ddf131855ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:40.503216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba762c-0e86-4eae-bbc8-5ddf131855ab admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.503532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba762c-0e86-4eae-bbc8-5ddf131855ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.503812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba762c-0e86-4eae-bbc8-5ddf131855ab admin admin] PATH_INFO: `/v3/projects/a9f8df8d077c496d996ea884ce3fb12a/users/d805e6f407a5432aa2c2fdf0c63f5256/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.506576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba762c-0e86-4eae-bbc8-5ddf131855ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9f8df8d077c496d996ea884ce3fb12a, user_id=d805e6f407a5432aa2c2fdf0c63f5256, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:40.509465 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba762c-0e86-4eae-bbc8-5ddf131855ab admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:40.522292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1049/2085] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:40 2022] PUT /identity/v3/projects/a9f8df8d077c496d996ea884ce3fb12a/users/d805e6f407a5432aa2c2fdf0c63f5256/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:40.530436 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba762c-0e86-4eae-bbc8-5ddf131855ab admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:40.564734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1c992d87-ed0b-4337-9a49-80b44727d04d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd819e39f5a0466296d8057328681125', 'request_id': 'req-1c992d87-ed0b-4337-9a49-80b44727d04d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:40.573227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1c992d87-ed0b-4337-9a49-80b44727d04d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd819e39f5a0466296d8057328681125', 'request_id': 'req-1c992d87-ed0b-4337-9a49-80b44727d04d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:40.578980 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1c992d87-ed0b-4337-9a49-80b44727d04d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd819e39f5a0466296d8057328681125', 'request_id': 'req-1c992d87-ed0b-4337-9a49-80b44727d04d'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:40.591226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba762c-0e86-4eae-bbc8-5ddf131855ab admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:40.619862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:40.656694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7cdbf-8ee4-46e5-be47-b09256d50080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:40.657325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7cdbf-8ee4-46e5-be47-b09256d50080 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.657611 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7cdbf-8ee4-46e5-be47-b09256d50080 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.657884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7cdbf-8ee4-46e5-be47-b09256d50080 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.658612 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7cdbf-8ee4-46e5-be47-b09256d50080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:40.660179 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1c992d87-ed0b-4337-9a49-80b44727d04d admin admin] The token cache is being invalidate because project cd819e39f5a0466296d8057328681125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:40.660392 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7cdbf-8ee4-46e5-be47-b09256d50080 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:40.660522 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1c992d87-ed0b-4337-9a49-80b44727d04d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:40.661316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1c992d87-ed0b-4337-9a49-80b44727d04d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:40.663800 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1037/2086] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:40 2022] DELETE /identity/v3/projects/cd819e39f5a0466296d8057328681125 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:40.666588 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d37718-76d9-4b4a-954a-fe70ef096605 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.675298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d37718-76d9-4b4a-954a-fe70ef096605 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.675298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d37718-76d9-4b4a-954a-fe70ef096605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.675584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1050/2087] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:40.675584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de719efb-2425-4b01-add3-a281522f4045 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.675584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de719efb-2425-4b01-add3-a281522f4045 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.675584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de719efb-2425-4b01-add3-a281522f4045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.698862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-de719efb-2425-4b01-add3-a281522f4045 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:40.700116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-de719efb-2425-4b01-add3-a281522f4045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:40.707770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-d1d37718-76d9-4b4a-954a-fe70ef096605 None None] MFA Rules not processed for user `d805e6f407a5432aa2c2fdf0c63f5256`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:40.709481 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d1d37718-76d9-4b4a-954a-fe70ef096605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:40.709696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1051/2088] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:40.710760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de719efb-2425-4b01-add3-a281522f4045 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:40.712168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-de719efb-2425-4b01-add3-a281522f4045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:40.726695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de719efb-2425-4b01-add3-a281522f4045 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:40.731848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3f9b3-371c-4933-9dce-ff25d1a55e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:40.732858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3f9b3-371c-4933-9dce-ff25d1a55e3b admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.732858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3f9b3-371c-4933-9dce-ff25d1a55e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.733349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3f9b3-371c-4933-9dce-ff25d1a55e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.733838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3f9b3-371c-4933-9dce-ff25d1a55e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:40.735417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3f9b3-371c-4933-9dce-ff25d1a55e3b admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:40.750127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1052/2089] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:40.751089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba73b4-ff5a-4a7a-974d-925b40381a9b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.751716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba73b4-ff5a-4a7a-974d-925b40381a9b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.752294 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba73b4-ff5a-4a7a-974d-925b40381a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.767353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:40.774820 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-dfba73b4-ff5a-4a7a-974d-925b40381a9b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:40.778761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-dfba73b4-ff5a-4a7a-974d-925b40381a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:40.784438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1053/2090] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:40.785485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba73b4-ff5a-4a7a-974d-925b40381a9b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:40.801916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba73b4-ff5a-4a7a-974d-925b40381a9b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:40.810998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1038/2091] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:40.810998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d37718-76d9-4b4a-954a-fe70ef096605 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:40.811288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea05baf-d066-40cc-9f8f-8fe9bee381de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:40.811288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05baf-d066-40cc-9f8f-8fe9bee381de admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.811288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05baf-d066-40cc-9f8f-8fe9bee381de admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.811288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05baf-d066-40cc-9f8f-8fe9bee381de admin admin] PATH_INFO: `/v3/projects/450b6bbee14b4c388918e127866589cc/users/04d99afe6d264f06be660bb78f9bceba/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.828290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d37718-76d9-4b4a-954a-fe70ef096605 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:40.829487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea05baf-d066-40cc-9f8f-8fe9bee381de admin admin] RBAC: Authorizing `identity:create_grant(project_id=450b6bbee14b4c388918e127866589cc, user_id=04d99afe6d264f06be660bb78f9bceba, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:40.832255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea05baf-d066-40cc-9f8f-8fe9bee381de admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:40.842484 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5a031-4dc8-4dc8-a696-be3b8377f281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:40.843122 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5a031-4dc8-4dc8-a696-be3b8377f281 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.843407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5a031-4dc8-4dc8-a696-be3b8377f281 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.843727 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5a031-4dc8-4dc8-a696-be3b8377f281 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.844432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5a031-4dc8-4dc8-a696-be3b8377f281 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:40.844737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5a031-4dc8-4dc8-a696-be3b8377f281 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:40.846351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5a031-4dc8-4dc8-a696-be3b8377f281 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:40.859270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1039/2092] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:06:40.860213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5a031-4dc8-4dc8-a696-be3b8377f281 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:40.861850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3cf5a031-4dc8-4dc8-a696-be3b8377f281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:40.922129 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5a031-4dc8-4dc8-a696-be3b8377f281 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:40.925969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae8c1d-904e-4944-8105-6ce8f1c106e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:40.926581 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae8c1d-904e-4944-8105-6ce8f1c106e9 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:40.926983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae8c1d-904e-4944-8105-6ce8f1c106e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:40.927266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae8c1d-904e-4944-8105-6ce8f1c106e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:40.931050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e5ae8c1d-904e-4944-8105-6ce8f1c106e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:40.959931 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1054/2093] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:40 2022] PUT /identity/v3/projects/450b6bbee14b4c388918e127866589cc/users/04d99afe6d264f06be660bb78f9bceba/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:40.969690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea05baf-d066-40cc-9f8f-8fe9bee381de admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:41.034772 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae8c1d-904e-4944-8105-6ce8f1c106e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:41.037796 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae8c1d-904e-4944-8105-6ce8f1c106e9 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:41.047307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea05baf-d066-40cc-9f8f-8fe9bee381de admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:41.048966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1040/2094] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:41.050922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85689283-08f2-46ea-9673-ece5e9d60153 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:41.051218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85689283-08f2-46ea-9673-ece5e9d60153 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:41.051603 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85689283-08f2-46ea-9673-ece5e9d60153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:41.055037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e728879-c1d1-48d4-8862-b8d69620ce31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:41.055951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e728879-c1d1-48d4-8862-b8d69620ce31 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:41.056454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e728879-c1d1-48d4-8862-b8d69620ce31 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:41.056830 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e728879-c1d1-48d4-8862-b8d69620ce31 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:41.057633 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e728879-c1d1-48d4-8862-b8d69620ce31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:41.059984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e728879-c1d1-48d4-8862-b8d69620ce31 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:41.081994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:41.113349 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-85689283-08f2-46ea-9673-ece5e9d60153 None None] MFA Rules not processed for user `04d99afe6d264f06be660bb78f9bceba`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:41.116093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-85689283-08f2-46ea-9673-ece5e9d60153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:41.200054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1055/2095] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:40 2022] POST /identity/v3/projects => generated 373 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:41.200766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1041/2096] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:41.205998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e728879-c1d1-48d4-8862-b8d69620ce31 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:41.213240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85689283-08f2-46ea-9673-ece5e9d60153 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:41.214847 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-85689283-08f2-46ea-9673-ece5e9d60153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:41.228996 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:41.297891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85689283-08f2-46ea-9673-ece5e9d60153 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:41.303887 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e728879-c1d1-48d4-8862-b8d69620ce31 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:41.308926 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa41539-b9ab-4869-9b75-b2bb9cd8df06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:41.309569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa41539-b9ab-4869-9b75-b2bb9cd8df06 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:41.309849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa41539-b9ab-4869-9b75-b2bb9cd8df06 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:41.310122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa41539-b9ab-4869-9b75-b2bb9cd8df06 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:41.310974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa41539-b9ab-4869-9b75-b2bb9cd8df06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:41.313457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa41539-b9ab-4869-9b75-b2bb9cd8df06 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:41.365317 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84ca6c-378b-4ec7-8f3f-c7d2146dd0b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:41.366184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84ca6c-378b-4ec7-8f3f-c7d2146dd0b4 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:41.366536 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84ca6c-378b-4ec7-8f3f-c7d2146dd0b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:41.366884 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84ca6c-378b-4ec7-8f3f-c7d2146dd0b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:41.403584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1056/2097] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:41 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:41.408745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa41539-b9ab-4869-9b75-b2bb9cd8df06 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:41.420496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa41539-b9ab-4869-9b75-b2bb9cd8df06 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:41.425457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbeaef2-f0ba-4f8d-a8af-2e17870bd3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:41.438941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeaef2-f0ba-4f8d-a8af-2e17870bd3e4 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:41.438941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeaef2-f0ba-4f8d-a8af-2e17870bd3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:41.438941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeaef2-f0ba-4f8d-a8af-2e17870bd3e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:41.438941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbeaef2-f0ba-4f8d-a8af-2e17870bd3e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:41.438941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbeaef2-f0ba-4f8d-a8af-2e17870bd3e4 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:41.439564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84ca6c-378b-4ec7-8f3f-c7d2146dd0b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:41.439564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84ca6c-378b-4ec7-8f3f-c7d2146dd0b4 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:41.440609 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1057/2098] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:06:41.445374 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbeaef2-f0ba-4f8d-a8af-2e17870bd3e4 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:41.446223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1042/2099] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:41 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:41.455785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbeaef2-f0ba-4f8d-a8af-2e17870bd3e4 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:41.461720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8cac7-9dc8-45a8-adc9-cdaaaa021499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:41.462334 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8cac7-9dc8-45a8-adc9-cdaaaa021499 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:41.462615 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8cac7-9dc8-45a8-adc9-cdaaaa021499 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:41.462887 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8cac7-9dc8-45a8-adc9-cdaaaa021499 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:41.463608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8cac7-9dc8-45a8-adc9-cdaaaa021499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:41.465201 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8cac7-9dc8-45a8-adc9-cdaaaa021499 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:41.477007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1058/2100] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:06:41.483432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84ca6c-378b-4ec7-8f3f-c7d2146dd0b4 service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:41.511455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84ca6c-378b-4ec7-8f3f-c7d2146dd0b4 service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:41.516747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f0c8d-04dd-412b-8254-4e4ffd7b2387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:41.517535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f0c8d-04dd-412b-8254-4e4ffd7b2387 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:41.517919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f0c8d-04dd-412b-8254-4e4ffd7b2387 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:41.518296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f0c8d-04dd-412b-8254-4e4ffd7b2387 admin admin] PATH_INFO: `/v3/projects/306f7bbc2933454487375b44207a2daf/users/de883f07b9854ff692a3cbbd46ea9af2/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:41.541097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f0c8d-04dd-412b-8254-4e4ffd7b2387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=306f7bbc2933454487375b44207a2daf, user_id=de883f07b9854ff692a3cbbd46ea9af2, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:41.560851 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f0c8d-04dd-412b-8254-4e4ffd7b2387 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:41.867807 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1043/2101] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:41 2022] PUT /identity/v3/projects/306f7bbc2933454487375b44207a2daf/users/de883f07b9854ff692a3cbbd46ea9af2/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:41.878768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4da11-9ca3-49d3-a0d0-9292a769b200 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:41.879355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4da11-9ca3-49d3-a0d0-9292a769b200 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:41.879355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4da11-9ca3-49d3-a0d0-9292a769b200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:41.908853 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-41e4da11-9ca3-49d3-a0d0-9292a769b200 None None] MFA Rules not processed for user `de883f07b9854ff692a3cbbd46ea9af2`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:41.910102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-41e4da11-9ca3-49d3-a0d0-9292a769b200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:41.983980 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1059/2102] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:42.476384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f0c8d-04dd-412b-8254-4e4ffd7b2387 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:42.534310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f0c8d-04dd-412b-8254-4e4ffd7b2387 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:42.540485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4341d-2033-4e77-a4cb-a8b774f1f66b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:42.540485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4341d-2033-4e77-a4cb-a8b774f1f66b service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:42.540485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4341d-2033-4e77-a4cb-a8b774f1f66b service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:42.540485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4341d-2033-4e77-a4cb-a8b774f1f66b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:42.547216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:42.550121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4341d-2033-4e77-a4cb-a8b774f1f66b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:42.551751 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4341d-2033-4e77-a4cb-a8b774f1f66b service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:42.564011 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1044/2103] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:42.591410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4da11-9ca3-49d3-a0d0-9292a769b200 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:42.670974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:42.684631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4da11-9ca3-49d3-a0d0-9292a769b200 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:42.691504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8993460-7baf-4b7d-95ce-3701d1f36481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:42.700438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8993460-7baf-4b7d-95ce-3701d1f36481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:42.700438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8993460-7baf-4b7d-95ce-3701d1f36481 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:42.700438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8993460-7baf-4b7d-95ce-3701d1f36481 admin admin] PATH_INFO: `/v3/users/de883f07b9854ff692a3cbbd46ea9af2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:42.700438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8993460-7baf-4b7d-95ce-3701d1f36481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de883f07b9854ff692a3cbbd46ea9af2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:42.700438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8993460-7baf-4b7d-95ce-3701d1f36481 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:42.955028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c8993460-7baf-4b7d-95ce-3701d1f36481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de883f07b9854ff692a3cbbd46ea9af2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:42.963064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c8993460-7baf-4b7d-95ce-3701d1f36481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de883f07b9854ff692a3cbbd46ea9af2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:43.052873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c8993460-7baf-4b7d-95ce-3701d1f36481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de883f07b9854ff692a3cbbd46ea9af2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:43.067153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c8993460-7baf-4b7d-95ce-3701d1f36481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de883f07b9854ff692a3cbbd46ea9af2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:43.069706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c8993460-7baf-4b7d-95ce-3701d1f36481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de883f07b9854ff692a3cbbd46ea9af2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:43.074712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1060/2104] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:42 2022] DELETE /identity/v3/users/de883f07b9854ff692a3cbbd46ea9af2 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:43.079270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8f690-f6c1-4a4f-8067-b7cd6976cc7c None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:43.079584 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8f690-f6c1-4a4f-8067-b7cd6976cc7c None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:43.079869 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8f690-f6c1-4a4f-8067-b7cd6976cc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:43.099732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-2fd8f690-f6c1-4a4f-8067-b7cd6976cc7c None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:43.101246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-2fd8f690-f6c1-4a4f-8067-b7cd6976cc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:43.165054 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1045/2105] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:43.183703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8993460-7baf-4b7d-95ce-3701d1f36481 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:43.183703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c8993460-7baf-4b7d-95ce-3701d1f36481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:43.239834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8993460-7baf-4b7d-95ce-3701d1f36481 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:43.243603 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631e8b4-6ae9-478e-ba4c-525c0a842903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:43.244241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631e8b4-6ae9-478e-ba4c-525c0a842903 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:43.244527 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631e8b4-6ae9-478e-ba4c-525c0a842903 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:43.244806 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631e8b4-6ae9-478e-ba4c-525c0a842903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:43.255371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631e8b4-6ae9-478e-ba4c-525c0a842903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:43.257850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631e8b4-6ae9-478e-ba4c-525c0a842903 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:43.272052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1061/2106] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:06:43.761224 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8f690-f6c1-4a4f-8067-b7cd6976cc7c None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:43.762741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-2fd8f690-f6c1-4a4f-8067-b7cd6976cc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:43.788383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8f690-f6c1-4a4f-8067-b7cd6976cc7c None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:43.802427 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:43.803197 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:43.803495 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:43.803864 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb admin admin] PATH_INFO: `/v3/projects/306f7bbc2933454487375b44207a2daf` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:43.805137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=306f7bbc2933454487375b44207a2daf)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:43.807825 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:44.156475 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '306f7bbc2933454487375b44207a2daf', 'request_id': 'req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:44.165111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '306f7bbc2933454487375b44207a2daf', 'request_id': 'req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:44.172059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '306f7bbc2933454487375b44207a2daf', 'request_id': 'req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:44.282641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb admin admin] The token cache is being invalidate because project 306f7bbc2933454487375b44207a2daf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:44.284051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:44.284716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-f73dd82d-cfa9-4aac-97a7-77c86347a5eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:44.286443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1046/2107] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:43 2022] DELETE /identity/v3/projects/306f7bbc2933454487375b44207a2daf => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:44.292649 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80583d-a198-49a6-a0af-a87362f85816 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:44.293607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80583d-a198-49a6-a0af-a87362f85816 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:44.296324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80583d-a198-49a6-a0af-a87362f85816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:44.322396 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-dd80583d-a198-49a6-a0af-a87362f85816 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:44.323865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-dd80583d-a198-49a6-a0af-a87362f85816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:44.394041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1062/2108] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:44.464732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cc3fd-261e-467f-b96a-261bdade8b15 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:44.465024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cc3fd-261e-467f-b96a-261bdade8b15 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:44.465309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cc3fd-261e-467f-b96a-261bdade8b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:44.485868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-8c3cc3fd-261e-467f-b96a-261bdade8b15 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:44.487224 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8c3cc3fd-261e-467f-b96a-261bdade8b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:44.495277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1047/2109] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:44.500025 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80583d-a198-49a6-a0af-a87362f85816 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:44.510570 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80583d-a198-49a6-a0af-a87362f85816 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:44.515238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c3b91-c3a6-4e15-a754-25bed42615c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:44.515953 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c3b91-c3a6-4e15-a754-25bed42615c6 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:44.516433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c3b91-c3a6-4e15-a754-25bed42615c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:44.516704 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c3b91-c3a6-4e15-a754-25bed42615c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:44.517395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c3b91-c3a6-4e15-a754-25bed42615c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:44.517701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c3b91-c3a6-4e15-a754-25bed42615c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:44.519448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c3b91-c3a6-4e15-a754-25bed42615c6 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:44.533823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1063/2110] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:06:44.538536 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cc3fd-261e-467f-b96a-261bdade8b15 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:44.548201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cc3fd-261e-467f-b96a-261bdade8b15 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:44.555838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c4619-6543-4f86-80e9-56e87eddad43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:44.556470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c4619-6543-4f86-80e9-56e87eddad43 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:44.556760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c4619-6543-4f86-80e9-56e87eddad43 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:44.557032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c4619-6543-4f86-80e9-56e87eddad43 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:44.557726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c4619-6543-4f86-80e9-56e87eddad43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:44.559812 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c4619-6543-4f86-80e9-56e87eddad43 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:44.668145 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1048/2111] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:44 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:06:44.672257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c3b91-c3a6-4e15-a754-25bed42615c6 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:44.677575 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:44.684864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c3b91-c3a6-4e15-a754-25bed42615c6 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:44.781391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14e1f7-7f29-4067-b899-3b84e869ddd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:44.782005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e1f7-7f29-4067-b899-3b84e869ddd9 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:44.782286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e1f7-7f29-4067-b899-3b84e869ddd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:44.782561 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e1f7-7f29-4067-b899-3b84e869ddd9 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:44.783230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14e1f7-7f29-4067-b899-3b84e869ddd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:44.785184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14e1f7-7f29-4067-b899-3b84e869ddd9 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:44.876515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1064/2112] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:44 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:44.881596 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c4619-6543-4f86-80e9-56e87eddad43 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:44.892457 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c4619-6543-4f86-80e9-56e87eddad43 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:44.903336 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c083fdc-a86c-48e3-a3d1-52f9b4da8150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:44.903336 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c083fdc-a86c-48e3-a3d1-52f9b4da8150 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:44.903336 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c083fdc-a86c-48e3-a3d1-52f9b4da8150 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:44.903336 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c083fdc-a86c-48e3-a3d1-52f9b4da8150 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:44.903832 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c083fdc-a86c-48e3-a3d1-52f9b4da8150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:44.903832 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c083fdc-a86c-48e3-a3d1-52f9b4da8150 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:44.912482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1049/2113] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:44.916611 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14e1f7-7f29-4067-b899-3b84e869ddd9 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:44.926739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14e1f7-7f29-4067-b899-3b84e869ddd9 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:44.932593 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d8f43-1440-4f3b-b7e6-3f19e0ac5c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:44.932969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d8f43-1440-4f3b-b7e6-3f19e0ac5c89 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:44.933248 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d8f43-1440-4f3b-b7e6-3f19e0ac5c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:44.933516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d8f43-1440-4f3b-b7e6-3f19e0ac5c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:44.934210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d8f43-1440-4f3b-b7e6-3f19e0ac5c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:44.935837 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d8f43-1440-4f3b-b7e6-3f19e0ac5c89 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:44.947747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1065/2114] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:44.951966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c083fdc-a86c-48e3-a3d1-52f9b4da8150 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:44.962343 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c083fdc-a86c-48e3-a3d1-52f9b4da8150 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:44.967190 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dbf6e-f097-460c-953c-6608ead515d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:44.967903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dbf6e-f097-460c-953c-6608ead515d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:44.968183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dbf6e-f097-460c-953c-6608ead515d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:44.968493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dbf6e-f097-460c-953c-6608ead515d9 admin admin] PATH_INFO: `/v3/projects/87391d9e4bdc48209437f1d786f376c1/users/9cb5ce208a2442c2b9b39ac92da6a78e/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:44.984456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748dbf6e-f097-460c-953c-6608ead515d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87391d9e4bdc48209437f1d786f376c1, user_id=9cb5ce208a2442c2b9b39ac92da6a78e, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:44.987110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748dbf6e-f097-460c-953c-6608ead515d9 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:45.095142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1050/2115] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:44 2022] PUT /identity/v3/projects/87391d9e4bdc48209437f1d786f376c1/users/9cb5ce208a2442c2b9b39ac92da6a78e/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:45.100488 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78608f5d-26e3-4e54-8637-3dd0c5493dc0 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:45.100765 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78608f5d-26e3-4e54-8637-3dd0c5493dc0 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:45.101040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78608f5d-26e3-4e54-8637-3dd0c5493dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:45.121849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-78608f5d-26e3-4e54-8637-3dd0c5493dc0 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:45.123057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-78608f5d-26e3-4e54-8637-3dd0c5493dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:45.183417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1066/2116] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:45.194163 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dbf6e-f097-460c-953c-6608ead515d9 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:45.196353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-748dbf6e-f097-460c-953c-6608ead515d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:45.215330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:45.259981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dbf6e-f097-460c-953c-6608ead515d9 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:45.266753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a98851-de7a-4673-8d16-2b74e67cfe0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:45.266753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a98851-de7a-4673-8d16-2b74e67cfe0b service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:45.266753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a98851-de7a-4673-8d16-2b74e67cfe0b service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:45.266753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a98851-de7a-4673-8d16-2b74e67cfe0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:45.277000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a98851-de7a-4673-8d16-2b74e67cfe0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:45.278352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a98851-de7a-4673-8d16-2b74e67cfe0b service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:45.290110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1051/2117] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:45.349490 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78608f5d-26e3-4e54-8637-3dd0c5493dc0 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:45.349490 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-78608f5d-26e3-4e54-8637-3dd0c5493dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:45.364370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78608f5d-26e3-4e54-8637-3dd0c5493dc0 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:45.369945 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2af01f-9ea9-41d7-a5c2-a4218b176cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:45.370621 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2af01f-9ea9-41d7-a5c2-a4218b176cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:45.370900 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2af01f-9ea9-41d7-a5c2-a4218b176cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:45.371171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2af01f-9ea9-41d7-a5c2-a4218b176cc7 admin admin] PATH_INFO: `/v3/users/0757c09591c2499fa07e17ba5a88f9d3` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:45.372822 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2af01f-9ea9-41d7-a5c2-a4218b176cc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0757c09591c2499fa07e17ba5a88f9d3)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:45.374759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2af01f-9ea9-41d7-a5c2-a4218b176cc7 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:45.654629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7d2af01f-9ea9-41d7-a5c2-a4218b176cc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0757c09591c2499fa07e17ba5a88f9d3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:45.664305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7d2af01f-9ea9-41d7-a5c2-a4218b176cc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0757c09591c2499fa07e17ba5a88f9d3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:45.730453 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7d2af01f-9ea9-41d7-a5c2-a4218b176cc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0757c09591c2499fa07e17ba5a88f9d3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:45.756949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:45.762786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7d2af01f-9ea9-41d7-a5c2-a4218b176cc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0757c09591c2499fa07e17ba5a88f9d3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:45.767655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-7d2af01f-9ea9-41d7-a5c2-a4218b176cc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0757c09591c2499fa07e17ba5a88f9d3'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:45.786633 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1067/2118] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:45 2022] DELETE /identity/v3/users/0757c09591c2499fa07e17ba5a88f9d3 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:45.788876 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93a4cb-1cf6-486e-b7b9-fefb0c2f774a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:45.796633 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93a4cb-1cf6-486e-b7b9-fefb0c2f774a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:45.796633 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93a4cb-1cf6-486e-b7b9-fefb0c2f774a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:45.839451 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-8f93a4cb-1cf6-486e-b7b9-fefb0c2f774a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:45.841018 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8f93a4cb-1cf6-486e-b7b9-fefb0c2f774a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:45.932842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1052/2119] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:45.943746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2af01f-9ea9-41d7-a5c2-a4218b176cc7 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:45.945574 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7d2af01f-9ea9-41d7-a5c2-a4218b176cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:46.038650 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2af01f-9ea9-41d7-a5c2-a4218b176cc7 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:46.043031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9436d-60d1-40f4-89cc-bf87076a1dce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:46.043439 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9436d-60d1-40f4-89cc-bf87076a1dce service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:46.043710 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9436d-60d1-40f4-89cc-bf87076a1dce service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:46.044068 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9436d-60d1-40f4-89cc-bf87076a1dce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:46.055851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9436d-60d1-40f4-89cc-bf87076a1dce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:46.057338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9436d-60d1-40f4-89cc-bf87076a1dce service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:46.069037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1068/2120] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:46.542094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93a4cb-1cf6-486e-b7b9-fefb0c2f774a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:46.544372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8f93a4cb-1cf6-486e-b7b9-fefb0c2f774a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:46.572561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93a4cb-1cf6-486e-b7b9-fefb0c2f774a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:46.580294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37855559-0116-4158-9b12-50c46dc62875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:46.581271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37855559-0116-4158-9b12-50c46dc62875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:46.581667 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37855559-0116-4158-9b12-50c46dc62875 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:46.582121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37855559-0116-4158-9b12-50c46dc62875 admin admin] PATH_INFO: `/v3/projects/ab09a0cd208b4b2b93bd2c292b8f4e1f` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:46.584074 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37855559-0116-4158-9b12-50c46dc62875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab09a0cd208b4b2b93bd2c292b8f4e1f)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:46.586367 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37855559-0116-4158-9b12-50c46dc62875 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:46.761916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-37855559-0116-4158-9b12-50c46dc62875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab09a0cd208b4b2b93bd2c292b8f4e1f', 'request_id': 'req-37855559-0116-4158-9b12-50c46dc62875'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:46.777700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-37855559-0116-4158-9b12-50c46dc62875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab09a0cd208b4b2b93bd2c292b8f4e1f', 'request_id': 'req-37855559-0116-4158-9b12-50c46dc62875'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:46.797859 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-37855559-0116-4158-9b12-50c46dc62875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab09a0cd208b4b2b93bd2c292b8f4e1f', 'request_id': 'req-37855559-0116-4158-9b12-50c46dc62875'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:46.901974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-37855559-0116-4158-9b12-50c46dc62875 admin admin] The token cache is being invalidate because project ab09a0cd208b4b2b93bd2c292b8f4e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:46.906527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-37855559-0116-4158-9b12-50c46dc62875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:46.909881 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-37855559-0116-4158-9b12-50c46dc62875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:46.913270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1053/2121] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:46 2022] DELETE /identity/v3/projects/ab09a0cd208b4b2b93bd2c292b8f4e1f => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:46.917278 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9436d-60d1-40f4-89cc-bf87076a1dce service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:46.929204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9436d-60d1-40f4-89cc-bf87076a1dce service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:47.041390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0764ac4-1402-43bf-8d05-b75e29f2d2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:47.042091 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0764ac4-1402-43bf-8d05-b75e29f2d2d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:47.042529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0764ac4-1402-43bf-8d05-b75e29f2d2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:47.042800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0764ac4-1402-43bf-8d05-b75e29f2d2d1 admin admin] PATH_INFO: `/v3/users/d805e6f407a5432aa2c2fdf0c63f5256` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:47.044249 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0764ac4-1402-43bf-8d05-b75e29f2d2d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d805e6f407a5432aa2c2fdf0c63f5256)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:47.046229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0764ac4-1402-43bf-8d05-b75e29f2d2d1 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:47.285216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c0764ac4-1402-43bf-8d05-b75e29f2d2d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd805e6f407a5432aa2c2fdf0c63f5256'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:47.296513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c0764ac4-1402-43bf-8d05-b75e29f2d2d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd805e6f407a5432aa2c2fdf0c63f5256'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:47.437280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c0764ac4-1402-43bf-8d05-b75e29f2d2d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd805e6f407a5432aa2c2fdf0c63f5256'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:47.452241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c0764ac4-1402-43bf-8d05-b75e29f2d2d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd805e6f407a5432aa2c2fdf0c63f5256'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:47.453252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c0764ac4-1402-43bf-8d05-b75e29f2d2d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd805e6f407a5432aa2c2fdf0c63f5256'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:47.461429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1069/2122] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:46 2022] DELETE /identity/v3/users/d805e6f407a5432aa2c2fdf0c63f5256 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:47.627040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8893a3d-66c1-44ad-8f17-737ae0b64585 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:47.627040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8893a3d-66c1-44ad-8f17-737ae0b64585 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:47.627040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8893a3d-66c1-44ad-8f17-737ae0b64585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:47.705914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-d8893a3d-66c1-44ad-8f17-737ae0b64585 None None] MFA Rules not processed for user `9cb5ce208a2442c2b9b39ac92da6a78e`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:47.707115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d8893a3d-66c1-44ad-8f17-737ae0b64585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:47.746063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:47.803062 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1054/2123] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:47.816851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0764ac4-1402-43bf-8d05-b75e29f2d2d1 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:47.821107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c0764ac4-1402-43bf-8d05-b75e29f2d2d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:47.843652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:47.899527 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0764ac4-1402-43bf-8d05-b75e29f2d2d1 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:47.905188 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad738b-7130-49d9-bce0-40a6abf7b443 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:47.906194 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad738b-7130-49d9-bce0-40a6abf7b443 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:47.906557 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad738b-7130-49d9-bce0-40a6abf7b443 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:47.906832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad738b-7130-49d9-bce0-40a6abf7b443 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:47.917991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad738b-7130-49d9-bce0-40a6abf7b443 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:47.920360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad738b-7130-49d9-bce0-40a6abf7b443 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:47.942538 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1070/2124] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:48.108434 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8893a3d-66c1-44ad-8f17-737ae0b64585 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:48.109994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d8893a3d-66c1-44ad-8f17-737ae0b64585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:48.307840 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8893a3d-66c1-44ad-8f17-737ae0b64585 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:48.322369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcd7ca-0d55-4f93-b547-fb132f36dd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:48.327286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcd7ca-0d55-4f93-b547-fb132f36dd0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:48.327286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcd7ca-0d55-4f93-b547-fb132f36dd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:48.327286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcd7ca-0d55-4f93-b547-fb132f36dd0b admin admin] PATH_INFO: `/v3/projects/a9f8df8d077c496d996ea884ce3fb12a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:48.327286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcd7ca-0d55-4f93-b547-fb132f36dd0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9f8df8d077c496d996ea884ce3fb12a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:48.327286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcd7ca-0d55-4f93-b547-fb132f36dd0b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:48.507704 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-effcd7ca-0d55-4f93-b547-fb132f36dd0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9f8df8d077c496d996ea884ce3fb12a', 'request_id': 'req-effcd7ca-0d55-4f93-b547-fb132f36dd0b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:48.524034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-effcd7ca-0d55-4f93-b547-fb132f36dd0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9f8df8d077c496d996ea884ce3fb12a', 'request_id': 'req-effcd7ca-0d55-4f93-b547-fb132f36dd0b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:48.539719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-effcd7ca-0d55-4f93-b547-fb132f36dd0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9f8df8d077c496d996ea884ce3fb12a', 'request_id': 'req-effcd7ca-0d55-4f93-b547-fb132f36dd0b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:48.639071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-effcd7ca-0d55-4f93-b547-fb132f36dd0b admin admin] The token cache is being invalidate because project a9f8df8d077c496d996ea884ce3fb12a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:48.639728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-effcd7ca-0d55-4f93-b547-fb132f36dd0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:48.640171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-effcd7ca-0d55-4f93-b547-fb132f36dd0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:48.642246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1055/2125] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:48 2022] DELETE /identity/v3/projects/a9f8df8d077c496d996ea884ce3fb12a => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:48.647247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4e90e-0c0e-4e55-9df5-f8f55d7f0126 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:48.647847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4e90e-0c0e-4e55-9df5-f8f55d7f0126 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:48.648743 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4e90e-0c0e-4e55-9df5-f8f55d7f0126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:48.675270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b1c4e90e-0c0e-4e55-9df5-f8f55d7f0126 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:48.676982 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b1c4e90e-0c0e-4e55-9df5-f8f55d7f0126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:48.764179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1071/2126] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:48.765608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462606e3-fc80-4054-9967-aa2c1723d3a6 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:48.767974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462606e3-fc80-4054-9967-aa2c1723d3a6 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:48.767974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462606e3-fc80-4054-9967-aa2c1723d3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:48.796263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-462606e3-fc80-4054-9967-aa2c1723d3a6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:48.796370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-462606e3-fc80-4054-9967-aa2c1723d3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:48.808242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1056/2127] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:48.812392 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4e90e-0c0e-4e55-9df5-f8f55d7f0126 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:48.824738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4e90e-0c0e-4e55-9df5-f8f55d7f0126 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:48.831401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af1846-bdd4-44be-9b68-459c0f0adae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:48.832282 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af1846-bdd4-44be-9b68-459c0f0adae0 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:48.832650 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af1846-bdd4-44be-9b68-459c0f0adae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:48.832996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af1846-bdd4-44be-9b68-459c0f0adae0 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:48.834373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af1846-bdd4-44be-9b68-459c0f0adae0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:48.834778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af1846-bdd4-44be-9b68-459c0f0adae0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:48.836912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af1846-bdd4-44be-9b68-459c0f0adae0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:48.860244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1072/2128] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:06:48.862450 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462606e3-fc80-4054-9967-aa2c1723d3a6 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:48.876135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462606e3-fc80-4054-9967-aa2c1723d3a6 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:48.882009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f03c4a-2be7-4c9f-abee-406d10ffdc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:48.882735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f03c4a-2be7-4c9f-abee-406d10ffdc68 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:48.883022 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f03c4a-2be7-4c9f-abee-406d10ffdc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:48.883425 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f03c4a-2be7-4c9f-abee-406d10ffdc68 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:48.884232 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f03c4a-2be7-4c9f-abee-406d10ffdc68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:48.886278 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f03c4a-2be7-4c9f-abee-406d10ffdc68 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:48.986583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1057/2129] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:48 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:48.991576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af1846-bdd4-44be-9b68-459c0f0adae0 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:49.002429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af1846-bdd4-44be-9b68-459c0f0adae0 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:49.063541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa0531-49c7-4fd3-8950-07edb4c62274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:49.064302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa0531-49c7-4fd3-8950-07edb4c62274 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:49.064573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa0531-49c7-4fd3-8950-07edb4c62274 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:49.064841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa0531-49c7-4fd3-8950-07edb4c62274 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:49.065507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa0531-49c7-4fd3-8950-07edb4c62274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:49.067420 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa0531-49c7-4fd3-8950-07edb4c62274 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:49.152942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1073/2130] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:48 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:06:49.158245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f03c4a-2be7-4c9f-abee-406d10ffdc68 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:49.179271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f03c4a-2be7-4c9f-abee-406d10ffdc68 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:49.179271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6030708-67ad-4874-a13e-ffa004f7beef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:49.179271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6030708-67ad-4874-a13e-ffa004f7beef admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:49.179271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6030708-67ad-4874-a13e-ffa004f7beef admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:49.179648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6030708-67ad-4874-a13e-ffa004f7beef admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:49.179648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6030708-67ad-4874-a13e-ffa004f7beef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:49.180464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6030708-67ad-4874-a13e-ffa004f7beef admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:49.191967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1058/2131] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:49.196356 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa0531-49c7-4fd3-8950-07edb4c62274 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:49.207276 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa0531-49c7-4fd3-8950-07edb4c62274 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:49.212668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc64858-ebd0-4e00-8844-85a4a64471ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:49.213364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc64858-ebd0-4e00-8844-85a4a64471ea admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:49.213682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc64858-ebd0-4e00-8844-85a4a64471ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:49.213956 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc64858-ebd0-4e00-8844-85a4a64471ea admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:49.214688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc64858-ebd0-4e00-8844-85a4a64471ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:49.216328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc64858-ebd0-4e00-8844-85a4a64471ea admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:49.231968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1074/2132] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:06:49.233248 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6030708-67ad-4874-a13e-ffa004f7beef admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:49.250210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6030708-67ad-4874-a13e-ffa004f7beef admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:49.257011 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff25042-9a81-48d0-8a5d-5377d7d5b4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:49.257697 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff25042-9a81-48d0-8a5d-5377d7d5b4e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:49.257992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff25042-9a81-48d0-8a5d-5377d7d5b4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:49.258307 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff25042-9a81-48d0-8a5d-5377d7d5b4e4 admin admin] PATH_INFO: `/v3/projects/06f44e6d3d164242a9107933b6144e2f/users/c718453b1b2b428dac800007f031de6b/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:49.275672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff25042-9a81-48d0-8a5d-5377d7d5b4e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f44e6d3d164242a9107933b6144e2f, user_id=c718453b1b2b428dac800007f031de6b, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:49.278306 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff25042-9a81-48d0-8a5d-5377d7d5b4e4 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:49.398174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1059/2133] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:49 2022] PUT /identity/v3/projects/06f44e6d3d164242a9107933b6144e2f/users/c718453b1b2b428dac800007f031de6b/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:49.410368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f004899-ad61-4052-a306-2d118f3bb097 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:49.411481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f004899-ad61-4052-a306-2d118f3bb097 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:49.412323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f004899-ad61-4052-a306-2d118f3bb097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:49.442700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-6f004899-ad61-4052-a306-2d118f3bb097 None None] MFA Rules not processed for user `c718453b1b2b428dac800007f031de6b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:49.443994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6f004899-ad61-4052-a306-2d118f3bb097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:49.519093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1075/2134] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:49.531223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff25042-9a81-48d0-8a5d-5377d7d5b4e4 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:49.532949 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9ff25042-9a81-48d0-8a5d-5377d7d5b4e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:49.563891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:49.598999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff25042-9a81-48d0-8a5d-5377d7d5b4e4 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:49.605100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb2663-db2e-4f9e-b0b5-2c48007ea2c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:49.605464 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2663-db2e-4f9e-b0b5-2c48007ea2c8 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:49.605748 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2663-db2e-4f9e-b0b5-2c48007ea2c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:49.606021 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2663-db2e-4f9e-b0b5-2c48007ea2c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:49.626462 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb2663-db2e-4f9e-b0b5-2c48007ea2c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:49.627981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb2663-db2e-4f9e-b0b5-2c48007ea2c8 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:49.637788 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1060/2135] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:49.892151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1660c-9dd6-419f-b590-2ccb69b30dca None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:49.892522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1660c-9dd6-419f-b590-2ccb69b30dca None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:49.895654 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1660c-9dd6-419f-b590-2ccb69b30dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:49.913885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-62f1660c-9dd6-419f-b590-2ccb69b30dca None None] MFA Rules not processed for user `9cb5ce208a2442c2b9b39ac92da6a78e`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:49.915090 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-62f1660c-9dd6-419f-b590-2ccb69b30dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:49.973240 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1076/2136] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:50.004554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:51.574512 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235530b8-b1d3-474e-81f9-fbd1498b3bf0 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:51.575204 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235530b8-b1d3-474e-81f9-fbd1498b3bf0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:51.575204 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235530b8-b1d3-474e-81f9-fbd1498b3bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:51.595759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-235530b8-b1d3-474e-81f9-fbd1498b3bf0 None None] MFA Rules not processed for user `2bf28a5c061c42668fc54a0c50af64ee`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:51.597071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-235530b8-b1d3-474e-81f9-fbd1498b3bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:51.669190 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1061/2137] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:53.326205 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1660c-9dd6-419f-b590-2ccb69b30dca None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:53.327750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-62f1660c-9dd6-419f-b590-2ccb69b30dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:53.396276 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1660c-9dd6-419f-b590-2ccb69b30dca None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:53.401070 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332b81d-f843-45a7-bfc8-377eb4a4977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:53.401744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332b81d-f843-45a7-bfc8-377eb4a4977a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:53.402020 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332b81d-f843-45a7-bfc8-377eb4a4977a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:53.402295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332b81d-f843-45a7-bfc8-377eb4a4977a admin admin] PATH_INFO: `/v3/users/04d99afe6d264f06be660bb78f9bceba` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:53.403507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332b81d-f843-45a7-bfc8-377eb4a4977a admin admin] RBAC: Authorizing `identity:delete_user(user_id=04d99afe6d264f06be660bb78f9bceba)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:53.405435 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332b81d-f843-45a7-bfc8-377eb4a4977a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:53.796993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d332b81d-f843-45a7-bfc8-377eb4a4977a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04d99afe6d264f06be660bb78f9bceba'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:53.806993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d332b81d-f843-45a7-bfc8-377eb4a4977a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04d99afe6d264f06be660bb78f9bceba'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:53.810934 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:53.872052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d332b81d-f843-45a7-bfc8-377eb4a4977a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04d99afe6d264f06be660bb78f9bceba'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:53.886304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d332b81d-f843-45a7-bfc8-377eb4a4977a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04d99afe6d264f06be660bb78f9bceba'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:53.887082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d332b81d-f843-45a7-bfc8-377eb4a4977a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04d99afe6d264f06be660bb78f9bceba'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:53.893181 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1077/2138] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:53 2022] DELETE /identity/v3/users/04d99afe6d264f06be660bb78f9bceba => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:53.897689 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12c7d3-b496-404a-bc68-c453fa68c767 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:53.898222 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12c7d3-b496-404a-bc68-c453fa68c767 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:53.898373 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12c7d3-b496-404a-bc68-c453fa68c767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:53.918833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ec12c7d3-b496-404a-bc68-c453fa68c767 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:53.920114 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ec12c7d3-b496-404a-bc68-c453fa68c767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:53.990328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1062/2139] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:54.000620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332b81d-f843-45a7-bfc8-377eb4a4977a admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:54.002461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d332b81d-f843-45a7-bfc8-377eb4a4977a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:54.064078 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332b81d-f843-45a7-bfc8-377eb4a4977a admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:54.067791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1f171-d441-4395-bb5d-42db856507de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:54.069052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1f171-d441-4395-bb5d-42db856507de service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:54.069323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1f171-d441-4395-bb5d-42db856507de service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:54.069596 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1f171-d441-4395-bb5d-42db856507de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:54.079211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc1f171-d441-4395-bb5d-42db856507de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:54.080679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc1f171-d441-4395-bb5d-42db856507de service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:54.090543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1078/2140] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:54.968915 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12c7d3-b496-404a-bc68-c453fa68c767 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:54.970891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ec12c7d3-b496-404a-bc68-c453fa68c767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:54.993579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12c7d3-b496-404a-bc68-c453fa68c767 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:54.999986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d071f-97ce-4174-8631-80a733bff712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:55.003304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d071f-97ce-4174-8631-80a733bff712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:55.003304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d071f-97ce-4174-8631-80a733bff712 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:55.003304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d071f-97ce-4174-8631-80a733bff712 admin admin] PATH_INFO: `/v3/projects/450b6bbee14b4c388918e127866589cc` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:55.003304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d071f-97ce-4174-8631-80a733bff712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=450b6bbee14b4c388918e127866589cc)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:55.004954 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d071f-97ce-4174-8631-80a733bff712 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:55.133749 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:55.140754 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-148d071f-97ce-4174-8631-80a733bff712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '450b6bbee14b4c388918e127866589cc', 'request_id': 'req-148d071f-97ce-4174-8631-80a733bff712'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:55.147944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-148d071f-97ce-4174-8631-80a733bff712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '450b6bbee14b4c388918e127866589cc', 'request_id': 'req-148d071f-97ce-4174-8631-80a733bff712'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:55.153740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-148d071f-97ce-4174-8631-80a733bff712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '450b6bbee14b4c388918e127866589cc', 'request_id': 'req-148d071f-97ce-4174-8631-80a733bff712'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:55.224572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-148d071f-97ce-4174-8631-80a733bff712 admin admin] The token cache is being invalidate because project 450b6bbee14b4c388918e127866589cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:55.224873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-148d071f-97ce-4174-8631-80a733bff712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:55.225631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-148d071f-97ce-4174-8631-80a733bff712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:55.227399 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1063/2141] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:54 2022] DELETE /identity/v3/projects/450b6bbee14b4c388918e127866589cc => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:55.233455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f420fa-6b62-4ada-b4b4-adfb69c66ea7 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:55.233961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f420fa-6b62-4ada-b4b4-adfb69c66ea7 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:55.234054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f420fa-6b62-4ada-b4b4-adfb69c66ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:55.261792 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-76f420fa-6b62-4ada-b4b4-adfb69c66ea7 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:55.263052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-76f420fa-6b62-4ada-b4b4-adfb69c66ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:55.340705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1079/2142] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:55.348200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b80443-8ca8-4ce0-a135-cfcdf8e951cc None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:55.348491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b80443-8ca8-4ce0-a135-cfcdf8e951cc None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:55.348778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b80443-8ca8-4ce0-a135-cfcdf8e951cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:55.370033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-58b80443-8ca8-4ce0-a135-cfcdf8e951cc None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:55.371537 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-58b80443-8ca8-4ce0-a135-cfcdf8e951cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:55.379566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1064/2143] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:55.384625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f420fa-6b62-4ada-b4b4-adfb69c66ea7 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:55.394958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f420fa-6b62-4ada-b4b4-adfb69c66ea7 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:55.400300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68d022-f9ef-4215-9305-66f0bfa64e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:55.401041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68d022-f9ef-4215-9305-66f0bfa64e4a admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:55.401269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68d022-f9ef-4215-9305-66f0bfa64e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:55.404860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb68d022-f9ef-4215-9305-66f0bfa64e4a admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:55.404860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68d022-f9ef-4215-9305-66f0bfa64e4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:55.404860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68d022-f9ef-4215-9305-66f0bfa64e4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:55.404860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb68d022-f9ef-4215-9305-66f0bfa64e4a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:55.417518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1080/2144] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:06:55.422684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b80443-8ca8-4ce0-a135-cfcdf8e951cc None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:55.433015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b80443-8ca8-4ce0-a135-cfcdf8e951cc None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:55.437957 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317124e-0926-44e8-85bb-e5374e210173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:55.438573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317124e-0926-44e8-85bb-e5374e210173 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:55.438857 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317124e-0926-44e8-85bb-e5374e210173 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:55.439205 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317124e-0926-44e8-85bb-e5374e210173 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:55.439976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317124e-0926-44e8-85bb-e5374e210173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:55.441875 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317124e-0926-44e8-85bb-e5374e210173 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:55.523529 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1065/2145] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:55 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:55.528735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68d022-f9ef-4215-9305-66f0bfa64e4a admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:55.540010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb68d022-f9ef-4215-9305-66f0bfa64e4a admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:55.612325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7d82a-3254-4eca-ae5a-7171f9ce5da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:55.612955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7d82a-3254-4eca-ae5a-7171f9ce5da0 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:55.613238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7d82a-3254-4eca-ae5a-7171f9ce5da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:55.613511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7d82a-3254-4eca-ae5a-7171f9ce5da0 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:55.614179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7d82a-3254-4eca-ae5a-7171f9ce5da0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:55.616451 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7d82a-3254-4eca-ae5a-7171f9ce5da0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:55.706085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:55.712767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1081/2146] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:55 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:55.716780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317124e-0926-44e8-85bb-e5374e210173 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:55.728130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317124e-0926-44e8-85bb-e5374e210173 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:55.732781 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127542ff-3b2a-40dc-be60-a2800491000a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:55.733405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127542ff-3b2a-40dc-be60-a2800491000a admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:55.733689 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127542ff-3b2a-40dc-be60-a2800491000a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:55.733970 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127542ff-3b2a-40dc-be60-a2800491000a admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:55.734669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127542ff-3b2a-40dc-be60-a2800491000a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:55.736326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127542ff-3b2a-40dc-be60-a2800491000a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:55.750862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1066/2147] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:55.755008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7d82a-3254-4eca-ae5a-7171f9ce5da0 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:55.765788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7d82a-3254-4eca-ae5a-7171f9ce5da0 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:55.770536 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68625908-83a5-4496-8947-c6d0311e0b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:55.771240 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68625908-83a5-4496-8947-c6d0311e0b05 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:55.771550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68625908-83a5-4496-8947-c6d0311e0b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:55.771856 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68625908-83a5-4496-8947-c6d0311e0b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:55.772644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68625908-83a5-4496-8947-c6d0311e0b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:55.774267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68625908-83a5-4496-8947-c6d0311e0b05 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:55.788712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1082/2148] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:55.793182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127542ff-3b2a-40dc-be60-a2800491000a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:55.804458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127542ff-3b2a-40dc-be60-a2800491000a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:55.809108 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34dc89-63a0-46af-8d0f-a86a9c43bfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:55.809911 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34dc89-63a0-46af-8d0f-a86a9c43bfd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:55.810202 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34dc89-63a0-46af-8d0f-a86a9c43bfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:55.810486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34dc89-63a0-46af-8d0f-a86a9c43bfd8 admin admin] PATH_INFO: `/v3/projects/2995c91727ba49b39d52b8af8ff2407b/users/135998a9e33646d4aeb8824547291056/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:55.831567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34dc89-63a0-46af-8d0f-a86a9c43bfd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2995c91727ba49b39d52b8af8ff2407b, user_id=135998a9e33646d4aeb8824547291056, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:55.837226 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34dc89-63a0-46af-8d0f-a86a9c43bfd8 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:56.035498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1067/2149] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:55 2022] PUT /identity/v3/projects/2995c91727ba49b39d52b8af8ff2407b/users/135998a9e33646d4aeb8824547291056/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:56.047344 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370977f-9f48-47f9-8268-7d27fd56c758 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:56.047886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370977f-9f48-47f9-8268-7d27fd56c758 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:56.048264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370977f-9f48-47f9-8268-7d27fd56c758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:56.101921 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-a370977f-9f48-47f9-8268-7d27fd56c758 None None] MFA Rules not processed for user `135998a9e33646d4aeb8824547291056`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:56.103267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a370977f-9f48-47f9-8268-7d27fd56c758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:56.197696 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1083/2150] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:56.210135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34dc89-63a0-46af-8d0f-a86a9c43bfd8 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:56.212418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8c34dc89-63a0-46af-8d0f-a86a9c43bfd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:56.386274 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34dc89-63a0-46af-8d0f-a86a9c43bfd8 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:56.386274 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5794e-201b-4042-b4d6-4e9f70b73f7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:56.386996 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5794e-201b-4042-b4d6-4e9f70b73f7b service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:56.387165 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5794e-201b-4042-b4d6-4e9f70b73f7b service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:56.387446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5794e-201b-4042-b4d6-4e9f70b73f7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:56.397934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5794e-201b-4042-b4d6-4e9f70b73f7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:56.399739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5794e-201b-4042-b4d6-4e9f70b73f7b service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:56.422425 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1068/2151] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:06:57.122627 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370977f-9f48-47f9-8268-7d27fd56c758 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:57.124404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a370977f-9f48-47f9-8268-7d27fd56c758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:57.216992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370977f-9f48-47f9-8268-7d27fd56c758 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:57.222318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b1230-a42e-45eb-a18a-94051e1c554f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:57.223055 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b1230-a42e-45eb-a18a-94051e1c554f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:57.223809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b1230-a42e-45eb-a18a-94051e1c554f admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:57.223809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b1230-a42e-45eb-a18a-94051e1c554f admin admin] PATH_INFO: `/v3/users/d03d43d573ae4f3d8dbecc99ef6824b0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:57.226880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b1230-a42e-45eb-a18a-94051e1c554f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d03d43d573ae4f3d8dbecc99ef6824b0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:57.227293 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b1230-a42e-45eb-a18a-94051e1c554f admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:57.301406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5794e-201b-4042-b4d6-4e9f70b73f7b service nova] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:57.317318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5794e-201b-4042-b4d6-4e9f70b73f7b service nova] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:57.335658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6272105-9cbc-4300-be76-91505c4db558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:57.336693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6272105-9cbc-4300-be76-91505c4db558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:57.337135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6272105-9cbc-4300-be76-91505c4db558 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:57.337570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6272105-9cbc-4300-be76-91505c4db558 admin admin] PATH_INFO: `/v3/users/135998a9e33646d4aeb8824547291056` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:57.339648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6272105-9cbc-4300-be76-91505c4db558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=135998a9e33646d4aeb8824547291056)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:57.342220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6272105-9cbc-4300-be76-91505c4db558 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:57.546561 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-567b1230-a42e-45eb-a18a-94051e1c554f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd03d43d573ae4f3d8dbecc99ef6824b0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:57.558521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-567b1230-a42e-45eb-a18a-94051e1c554f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd03d43d573ae4f3d8dbecc99ef6824b0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:57.617115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:57.618987 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e6272105-9cbc-4300-be76-91505c4db558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '135998a9e33646d4aeb8824547291056'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:57.619801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e6272105-9cbc-4300-be76-91505c4db558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '135998a9e33646d4aeb8824547291056'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:57.620482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e6272105-9cbc-4300-be76-91505c4db558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '135998a9e33646d4aeb8824547291056'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:57.629213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e6272105-9cbc-4300-be76-91505c4db558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '135998a9e33646d4aeb8824547291056'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:57.657228 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e6272105-9cbc-4300-be76-91505c4db558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '135998a9e33646d4aeb8824547291056'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:57.660495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-567b1230-a42e-45eb-a18a-94051e1c554f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd03d43d573ae4f3d8dbecc99ef6824b0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:57.681390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:57.683052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-567b1230-a42e-45eb-a18a-94051e1c554f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd03d43d573ae4f3d8dbecc99ef6824b0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:57.685171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-567b1230-a42e-45eb-a18a-94051e1c554f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd03d43d573ae4f3d8dbecc99ef6824b0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:57.691606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1084/2152] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:57 2022] DELETE /identity/v3/users/d03d43d573ae4f3d8dbecc99ef6824b0 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:57.698617 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d86a-2759-45ae-b293-f639448468c8 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:57.698989 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d86a-2759-45ae-b293-f639448468c8 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:57.699273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d86a-2759-45ae-b293-f639448468c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:57.720742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-5cf0d86a-2759-45ae-b293-f639448468c8 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:57.722080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5cf0d86a-2759-45ae-b293-f639448468c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:57.787761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1069/2153] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:57 2022] DELETE /identity/v3/users/135998a9e33646d4aeb8824547291056 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:57.792897 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242b26d-2e51-488f-9e9b-bdec15b8715a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:57.793512 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242b26d-2e51-488f-9e9b-bdec15b8715a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:57.794195 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242b26d-2e51-488f-9e9b-bdec15b8715a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:57.801223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1085/2154] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:57.812054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d86a-2759-45ae-b293-f639448468c8 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:57.815097 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5cf0d86a-2759-45ae-b293-f639448468c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:57.841243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-3242b26d-2e51-488f-9e9b-bdec15b8715a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:57.843129 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3242b26d-2e51-488f-9e9b-bdec15b8715a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:57.886050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d86a-2759-45ae-b293-f639448468c8 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:57.925937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1070/2155] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:57.936213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242b26d-2e51-488f-9e9b-bdec15b8715a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:57.937699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3242b26d-2e51-488f-9e9b-bdec15b8715a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:57.946583 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9aedf-3941-4ec5-a052-91e444fb7ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:57.947252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9aedf-3941-4ec5-a052-91e444fb7ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:57.947573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9aedf-3941-4ec5-a052-91e444fb7ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:57.947854 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9aedf-3941-4ec5-a052-91e444fb7ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:57.962313 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9aedf-3941-4ec5-a052-91e444fb7ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:57.963660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9aedf-3941-4ec5-a052-91e444fb7ca1 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:57.975641 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1086/2156] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:06:58.042345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242b26d-2e51-488f-9e9b-bdec15b8715a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:58.051031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b93e8a-3e85-4e84-b21b-11453f0386de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:58.052017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b93e8a-3e85-4e84-b21b-11453f0386de service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:58.052317 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b93e8a-3e85-4e84-b21b-11453f0386de service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:58.052674 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b93e8a-3e85-4e84-b21b-11453f0386de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:58.070638 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b93e8a-3e85-4e84-b21b-11453f0386de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:58.072126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b93e8a-3e85-4e84-b21b-11453f0386de service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:58.085130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1071/2157] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:06:58.468707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9aedf-3941-4ec5-a052-91e444fb7ca1 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:58.472073 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-52b9aedf-3941-4ec5-a052-91e444fb7ca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:58.492620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9aedf-3941-4ec5-a052-91e444fb7ca1 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:58.497904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85487e43-8cd1-447d-89e0-cfe31f3e44a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:58.498565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85487e43-8cd1-447d-89e0-cfe31f3e44a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:58.498849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85487e43-8cd1-447d-89e0-cfe31f3e44a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:58.499129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85487e43-8cd1-447d-89e0-cfe31f3e44a6 admin admin] PATH_INFO: `/v3/projects/c9cfcb9e78de43ae827f021047253214` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:58.500420 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85487e43-8cd1-447d-89e0-cfe31f3e44a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9cfcb9e78de43ae827f021047253214)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:58.502519 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85487e43-8cd1-447d-89e0-cfe31f3e44a6 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:58.561780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b93e8a-3e85-4e84-b21b-11453f0386de service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:58.563982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-42b93e8a-3e85-4e84-b21b-11453f0386de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:58.579730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b93e8a-3e85-4e84-b21b-11453f0386de service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:58.587406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6c046-02dc-4bc6-a66a-3db56ccd9596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:58.588115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6c046-02dc-4bc6-a66a-3db56ccd9596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:58.588419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6c046-02dc-4bc6-a66a-3db56ccd9596 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:58.588654 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6c046-02dc-4bc6-a66a-3db56ccd9596 admin admin] PATH_INFO: `/v3/projects/2995c91727ba49b39d52b8af8ff2407b` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:58.590243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6c046-02dc-4bc6-a66a-3db56ccd9596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2995c91727ba49b39d52b8af8ff2407b)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:58.592333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6c046-02dc-4bc6-a66a-3db56ccd9596 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:58.651131 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-85487e43-8cd1-447d-89e0-cfe31f3e44a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9cfcb9e78de43ae827f021047253214', 'request_id': 'req-85487e43-8cd1-447d-89e0-cfe31f3e44a6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:58.657029 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-85487e43-8cd1-447d-89e0-cfe31f3e44a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9cfcb9e78de43ae827f021047253214', 'request_id': 'req-85487e43-8cd1-447d-89e0-cfe31f3e44a6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:58.722471 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-85487e43-8cd1-447d-89e0-cfe31f3e44a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9cfcb9e78de43ae827f021047253214', 'request_id': 'req-85487e43-8cd1-447d-89e0-cfe31f3e44a6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:58.731386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-85e6c046-02dc-4bc6-a66a-3db56ccd9596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2995c91727ba49b39d52b8af8ff2407b', 'request_id': 'req-85e6c046-02dc-4bc6-a66a-3db56ccd9596'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:58.736052 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-85487e43-8cd1-447d-89e0-cfe31f3e44a6 admin admin] The token cache is being invalidate because project c9cfcb9e78de43ae827f021047253214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:58.736340 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-85487e43-8cd1-447d-89e0-cfe31f3e44a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:58.737017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-85487e43-8cd1-447d-89e0-cfe31f3e44a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:58.737874 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-85e6c046-02dc-4bc6-a66a-3db56ccd9596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2995c91727ba49b39d52b8af8ff2407b', 'request_id': 'req-85e6c046-02dc-4bc6-a66a-3db56ccd9596'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:58.738733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1087/2158] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:58 2022] DELETE /identity/v3/projects/c9cfcb9e78de43ae827f021047253214 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:06:58.743305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:58.744626 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b53e45-f23c-4f97-aa1f-881a714d6024 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:58.744770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-85e6c046-02dc-4bc6-a66a-3db56ccd9596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2995c91727ba49b39d52b8af8ff2407b', 'request_id': 'req-85e6c046-02dc-4bc6-a66a-3db56ccd9596'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:58.745063 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b53e45-f23c-4f97-aa1f-881a714d6024 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:58.745651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b53e45-f23c-4f97-aa1f-881a714d6024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:58.768055 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-49b53e45-f23c-4f97-aa1f-881a714d6024 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:58.769277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-49b53e45-f23c-4f97-aa1f-881a714d6024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:58.823608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-85e6c046-02dc-4bc6-a66a-3db56ccd9596 admin admin] The token cache is being invalidate because project 2995c91727ba49b39d52b8af8ff2407b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:06:58.823909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-85e6c046-02dc-4bc6-a66a-3db56ccd9596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:58.824579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-85e6c046-02dc-4bc6-a66a-3db56ccd9596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:06:58.826292 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1072/2159] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:06:58 2022] DELETE /identity/v3/projects/2995c91727ba49b39d52b8af8ff2407b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:58.831560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93dcecf-d32b-4011-be4d-225896c9cf9b None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:58.831891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93dcecf-d32b-4011-be4d-225896c9cf9b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:58.832456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93dcecf-d32b-4011-be4d-225896c9cf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:58.839505 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1088/2160] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:58.854264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-c93dcecf-d32b-4011-be4d-225896c9cf9b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:58.855357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c93dcecf-d32b-4011-be4d-225896c9cf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:58.861192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537b080-4ee5-4228-8d1f-b1c9929c5805 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:58.861483 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537b080-4ee5-4228-8d1f-b1c9929c5805 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:58.861758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537b080-4ee5-4228-8d1f-b1c9929c5805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:58.864793 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1073/2161] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:58.874155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda09b43-4896-4705-a7de-551cf5778c54 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:58.874384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda09b43-4896-4705-a7de-551cf5778c54 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:58.874849 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda09b43-4896-4705-a7de-551cf5778c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:58.885918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7537b080-4ee5-4228-8d1f-b1c9929c5805 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:58.887144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7537b080-4ee5-4228-8d1f-b1c9929c5805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:58.896458 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1089/2162] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:06:58.898125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-eda09b43-4896-4705-a7de-551cf5778c54 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:58.899304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-eda09b43-4896-4705-a7de-551cf5778c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:58.901076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537b080-4ee5-4228-8d1f-b1c9929c5805 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:58.910200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1074/2163] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:58.913731 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537b080-4ee5-4228-8d1f-b1c9929c5805 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:58.915778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda09b43-4896-4705-a7de-551cf5778c54 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:58.925266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda09b43-4896-4705-a7de-551cf5778c54 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:58.928644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f8397-d84b-488c-b077-e10f8460dedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:58.929329 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f8397-d84b-488c-b077-e10f8460dedd admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:58.929631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f8397-d84b-488c-b077-e10f8460dedd admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:58.929940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f8397-d84b-488c-b077-e10f8460dedd admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:58.931092 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f8397-d84b-488c-b077-e10f8460dedd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:58.931333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafcd1e-1141-4f63-84bf-1a072d80f7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:58.931439 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f8397-d84b-488c-b077-e10f8460dedd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:58.931964 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafcd1e-1141-4f63-84bf-1a072d80f7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:58.932175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafcd1e-1141-4f63-84bf-1a072d80f7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:58.932553 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafcd1e-1141-4f63-84bf-1a072d80f7c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:58.932726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f8397-d84b-488c-b077-e10f8460dedd admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:58.933395 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aafcd1e-1141-4f63-84bf-1a072d80f7c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:06:58.933767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aafcd1e-1141-4f63-84bf-1a072d80f7c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:58.935451 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aafcd1e-1141-4f63-84bf-1a072d80f7c1 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:58.936176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:58.949230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1090/2164] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:06:58.949429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1075/2165] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:06:58.952554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f8397-d84b-488c-b077-e10f8460dedd admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:58.953997 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafcd1e-1141-4f63-84bf-1a072d80f7c1 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:58.965238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f8397-d84b-488c-b077-e10f8460dedd admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:58.967588 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafcd1e-1141-4f63-84bf-1a072d80f7c1 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:58.971131 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c244e-ab6a-41b4-8333-f0dc461ba890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:58.972500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c244e-ab6a-41b4-8333-f0dc461ba890 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:58.972500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c244e-ab6a-41b4-8333-f0dc461ba890 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:58.972500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c244e-ab6a-41b4-8333-f0dc461ba890 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:58.972726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216515a4-fc39-4291-bba4-83d15c683d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:58.972975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c244e-ab6a-41b4-8333-f0dc461ba890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:58.973956 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216515a4-fc39-4291-bba4-83d15c683d6a admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:58.974246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216515a4-fc39-4291-bba4-83d15c683d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:58.974528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216515a4-fc39-4291-bba4-83d15c683d6a admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:58.975385 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216515a4-fc39-4291-bba4-83d15c683d6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:58.975568 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c244e-ab6a-41b4-8333-f0dc461ba890 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:58.977344 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216515a4-fc39-4291-bba4-83d15c683d6a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:59.090095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1091/2166] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:58 2022] POST /identity/v3/projects => generated 361 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:59.094442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c244e-ab6a-41b4-8333-f0dc461ba890 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:59.107596 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1076/2167] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:58 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:06:59.113573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216515a4-fc39-4291-bba4-83d15c683d6a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:59.168382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c244e-ab6a-41b4-8333-f0dc461ba890 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:59.187201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216515a4-fc39-4291-bba4-83d15c683d6a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:59.192491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452baec-e657-4c5e-9719-2cc3232a1ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:59.193994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452baec-e657-4c5e-9719-2cc3232a1ccb admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:59.194348 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452baec-e657-4c5e-9719-2cc3232a1ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:59.194618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452baec-e657-4c5e-9719-2cc3232a1ccb admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:59.195528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452baec-e657-4c5e-9719-2cc3232a1ccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:59.197418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452baec-e657-4c5e-9719-2cc3232a1ccb admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:59.230965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e561c21-1f34-441a-94ea-5d77fb094d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:59.231666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561c21-1f34-441a-94ea-5d77fb094d4b admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:59.231984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561c21-1f34-441a-94ea-5d77fb094d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:59.232265 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e561c21-1f34-441a-94ea-5d77fb094d4b admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:59.233022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e561c21-1f34-441a-94ea-5d77fb094d4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:59.235061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e561c21-1f34-441a-94ea-5d77fb094d4b admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:59.357249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1077/2168] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:59 2022] POST /identity/v3/users => generated 290 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:06:59.361414 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452baec-e657-4c5e-9719-2cc3232a1ccb admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:59.372687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452baec-e657-4c5e-9719-2cc3232a1ccb admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:59.377562 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb94364-2817-43e2-b488-1a411edaed0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:59.378221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb94364-2817-43e2-b488-1a411edaed0a admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:59.378525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb94364-2817-43e2-b488-1a411edaed0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:59.378804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb94364-2817-43e2-b488-1a411edaed0a admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:59.379595 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb94364-2817-43e2-b488-1a411edaed0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:59.384608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb94364-2817-43e2-b488-1a411edaed0a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:59.392544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1078/2169] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:06:59.396440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb94364-2817-43e2-b488-1a411edaed0a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:59.407624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb94364-2817-43e2-b488-1a411edaed0a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:59.407792 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1092/2170] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:06:59 2022] POST /identity/v3/users => generated 281 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:06:59.411025 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e561c21-1f34-441a-94ea-5d77fb094d4b admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:59.412669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff800bec-09a7-46a2-8bf2-9dab694e70fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:59.413283 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff800bec-09a7-46a2-8bf2-9dab694e70fe admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:59.413566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff800bec-09a7-46a2-8bf2-9dab694e70fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:59.413845 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff800bec-09a7-46a2-8bf2-9dab694e70fe admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:59.414578 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff800bec-09a7-46a2-8bf2-9dab694e70fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:59.416479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff800bec-09a7-46a2-8bf2-9dab694e70fe admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:59.421037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e561c21-1f34-441a-94ea-5d77fb094d4b admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:59.425908 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda6cda-c49c-4822-91f2-cf2c316f3cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:59.427553 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda6cda-c49c-4822-91f2-cf2c316f3cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:59.427553 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda6cda-c49c-4822-91f2-cf2c316f3cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:59.428131 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda6cda-c49c-4822-91f2-cf2c316f3cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:59.428508 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda6cda-c49c-4822-91f2-cf2c316f3cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:59.428606 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1079/2171] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:59.430125 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda6cda-c49c-4822-91f2-cf2c316f3cd6 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:59.432738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff800bec-09a7-46a2-8bf2-9dab694e70fe admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:59.444337 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1093/2172] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:06:59.444512 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff800bec-09a7-46a2-8bf2-9dab694e70fe admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:59.445726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda6cda-c49c-4822-91f2-cf2c316f3cd6 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:59.448196 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de64300-8e29-406d-836b-48b3484b7f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:59.448868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de64300-8e29-406d-836b-48b3484b7f49 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:59.449151 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de64300-8e29-406d-836b-48b3484b7f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:59.449431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de64300-8e29-406d-836b-48b3484b7f49 admin admin] PATH_INFO: `/v3/projects/8dcc2cf3cf2f49348683645556b0c2d7/users/16db51d64cd54ac5a6090b76729783cb/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:59.455723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda6cda-c49c-4822-91f2-cf2c316f3cd6 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:59.460778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4578c-0a54-4243-992c-e8d27962086a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:59.461172 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4578c-0a54-4243-992c-e8d27962086a admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:59.461452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4578c-0a54-4243-992c-e8d27962086a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:59.461765 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4578c-0a54-4243-992c-e8d27962086a admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:59.462409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4578c-0a54-4243-992c-e8d27962086a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:59.464125 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4578c-0a54-4243-992c-e8d27962086a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:59.464411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de64300-8e29-406d-836b-48b3484b7f49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dcc2cf3cf2f49348683645556b0c2d7, user_id=16db51d64cd54ac5a6090b76729783cb, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:59.467053 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de64300-8e29-406d-836b-48b3484b7f49 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:59.475449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1094/2173] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:06:59.479951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4578c-0a54-4243-992c-e8d27962086a admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:59.491316 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4578c-0a54-4243-992c-e8d27962086a admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:59.496319 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36bcef-e3e4-468d-aa81-8320fe6bab73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:59.497026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36bcef-e3e4-468d-aa81-8320fe6bab73 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:59.497323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36bcef-e3e4-468d-aa81-8320fe6bab73 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:59.497582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36bcef-e3e4-468d-aa81-8320fe6bab73 admin admin] PATH_INFO: `/v3/projects/03b62805a4c84aa4a9d57828b98015b6/users/c1d79e475c1d4283af49b8a1220e45aa/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:59.513292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36bcef-e3e4-468d-aa81-8320fe6bab73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b62805a4c84aa4a9d57828b98015b6, user_id=c1d79e475c1d4283af49b8a1220e45aa, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:59.516141 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36bcef-e3e4-468d-aa81-8320fe6bab73 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:59.562569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:59.620611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1080/2174] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:59 2022] PUT /identity/v3/projects/8dcc2cf3cf2f49348683645556b0c2d7/users/16db51d64cd54ac5a6090b76729783cb/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:59.632564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c74b4-74d8-4007-af24-d4eccbddcb2a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:59.633027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c74b4-74d8-4007-af24-d4eccbddcb2a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:59.633317 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c74b4-74d8-4007-af24-d4eccbddcb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:59.651358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1095/2175] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:06:59 2022] PUT /identity/v3/projects/03b62805a4c84aa4a9d57828b98015b6/users/c1d79e475c1d4283af49b8a1220e45aa/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:06:59.663300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b911e-7f06-4d75-a207-1fea6f77497d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:59.664604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b911e-7f06-4d75-a207-1fea6f77497d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:59.665081 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b911e-7f06-4d75-a207-1fea6f77497d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:59.678622 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-512c74b4-74d8-4007-af24-d4eccbddcb2a None None] MFA Rules not processed for user `16db51d64cd54ac5a6090b76729783cb`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:59.680237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-512c74b4-74d8-4007-af24-d4eccbddcb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:59.704244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-253b911e-7f06-4d75-a207-1fea6f77497d None None] MFA Rules not processed for user `c1d79e475c1d4283af49b8a1220e45aa`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:06:59.705470 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-253b911e-7f06-4d75-a207-1fea6f77497d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:59.728003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:06:59.792233 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1081/2176] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:59.797634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1096/2177] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:06:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:06:59.797800 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c74b4-74d8-4007-af24-d4eccbddcb2a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:59.839338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b911e-7f06-4d75-a207-1fea6f77497d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:59.841846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-253b911e-7f06-4d75-a207-1fea6f77497d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:06:59.884032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c74b4-74d8-4007-af24-d4eccbddcb2a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:59.892354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641be36a-561d-483e-bd8d-fad70f72d6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:59.892971 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641be36a-561d-483e-bd8d-fad70f72d6de admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:59.896907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641be36a-561d-483e-bd8d-fad70f72d6de admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:59.896907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641be36a-561d-483e-bd8d-fad70f72d6de admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:59.896907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641be36a-561d-483e-bd8d-fad70f72d6de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:59.896907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641be36a-561d-483e-bd8d-fad70f72d6de admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:59.918523 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b911e-7f06-4d75-a207-1fea6f77497d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:59.922493 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fc61a-a319-4e7a-ae4e-ac564dcab72f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:59.923482 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fc61a-a319-4e7a-ae4e-ac564dcab72f service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:59.923863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fc61a-a319-4e7a-ae4e-ac564dcab72f service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:59.924210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fc61a-a319-4e7a-ae4e-ac564dcab72f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:59.937966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fc61a-a319-4e7a-ae4e-ac564dcab72f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:59.939507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fc61a-a319-4e7a-ae4e-ac564dcab72f service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:06:59.950463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1097/2178] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:06:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:06:59.968697 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fc61a-a319-4e7a-ae4e-ac564dcab72f service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:06:59.980372 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fc61a-a319-4e7a-ae4e-ac564dcab72f service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:06:59.985621 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7f05b-f126-4802-9762-282b64b8553e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:06:59.986372 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7f05b-f126-4802-9762-282b64b8553e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:06:59.986874 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7f05b-f126-4802-9762-282b64b8553e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:06:59.987249 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7f05b-f126-4802-9762-282b64b8553e admin admin] PATH_INFO: `/v3/users/c1d79e475c1d4283af49b8a1220e45aa` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:06:59.988724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7f05b-f126-4802-9762-282b64b8553e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1d79e475c1d4283af49b8a1220e45aa)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:06:59.991579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7f05b-f126-4802-9762-282b64b8553e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:00.014692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1082/2179] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:06:59 2022] POST /identity/v3/projects => generated 381 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:00.018242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641be36a-561d-483e-bd8d-fad70f72d6de admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:00.041037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:00.102263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641be36a-561d-483e-bd8d-fad70f72d6de admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:00.110107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1bd34-09c1-410f-9d6b-9d1e3838750f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:00.111846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1bd34-09c1-410f-9d6b-9d1e3838750f admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:00.112738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1bd34-09c1-410f-9d6b-9d1e3838750f admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:00.113342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1bd34-09c1-410f-9d6b-9d1e3838750f admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:00.117465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1bd34-09c1-410f-9d6b-9d1e3838750f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:00.122183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1bd34-09c1-410f-9d6b-9d1e3838750f admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:00.223558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-07c7f05b-f126-4802-9762-282b64b8553e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1d79e475c1d4283af49b8a1220e45aa'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:00.231798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-07c7f05b-f126-4802-9762-282b64b8553e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1d79e475c1d4283af49b8a1220e45aa'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:00.247572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1083/2180] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:07:00 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:07:00.247572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1bd34-09c1-410f-9d6b-9d1e3838750f admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:00.260214 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1bd34-09c1-410f-9d6b-9d1e3838750f admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:00.265989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30446a69-09b1-4ec6-8db6-5907d3603204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:00.266511 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30446a69-09b1-4ec6-8db6-5907d3603204 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:00.266825 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30446a69-09b1-4ec6-8db6-5907d3603204 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:00.267101 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30446a69-09b1-4ec6-8db6-5907d3603204 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:00.267879 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30446a69-09b1-4ec6-8db6-5907d3603204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:00.269509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30446a69-09b1-4ec6-8db6-5907d3603204 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:00.281402 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1084/2181] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:00.285703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30446a69-09b1-4ec6-8db6-5907d3603204 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:00.296489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30446a69-09b1-4ec6-8db6-5907d3603204 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:00.301539 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20181cee-a8d1-44dc-9de8-750bbc305def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:00.302930 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20181cee-a8d1-44dc-9de8-750bbc305def admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:00.303755 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20181cee-a8d1-44dc-9de8-750bbc305def admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:00.303755 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20181cee-a8d1-44dc-9de8-750bbc305def admin admin] PATH_INFO: `/v3/projects/3f6393499e8c4bf28f13fde6aaea8f78/users/53eb1f17ca054eae8e7d547a057f9b8a/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:00.319528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20181cee-a8d1-44dc-9de8-750bbc305def admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f6393499e8c4bf28f13fde6aaea8f78, user_id=53eb1f17ca054eae8e7d547a057f9b8a, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:00.322314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20181cee-a8d1-44dc-9de8-750bbc305def admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:00.334517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-07c7f05b-f126-4802-9762-282b64b8553e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1d79e475c1d4283af49b8a1220e45aa'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:00.348298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-07c7f05b-f126-4802-9762-282b64b8553e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1d79e475c1d4283af49b8a1220e45aa'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:00.349217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-07c7f05b-f126-4802-9762-282b64b8553e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1d79e475c1d4283af49b8a1220e45aa'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:00.356001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1098/2182] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:06:59 2022] DELETE /identity/v3/users/c1d79e475c1d4283af49b8a1220e45aa => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:00.360652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b024e9-f6e8-49f5-aab0-f8660d0fa6cd None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:00.360947 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b024e9-f6e8-49f5-aab0-f8660d0fa6cd None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:00.361226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b024e9-f6e8-49f5-aab0-f8660d0fa6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:00.381540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-46b024e9-f6e8-49f5-aab0-f8660d0fa6cd None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:00.382957 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-46b024e9-f6e8-49f5-aab0-f8660d0fa6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:00.444202 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1085/2183] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:00 2022] PUT /identity/v3/projects/3f6393499e8c4bf28f13fde6aaea8f78/users/53eb1f17ca054eae8e7d547a057f9b8a/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:07:00.448397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20181cee-a8d1-44dc-9de8-750bbc305def admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:00.450132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-20181cee-a8d1-44dc-9de8-750bbc305def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:00.451361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1099/2184] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:07:00.463840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b024e9-f6e8-49f5-aab0-f8660d0fa6cd None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:00.465180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-46b024e9-f6e8-49f5-aab0-f8660d0fa6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:00.471606 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:00.528102 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20181cee-a8d1-44dc-9de8-750bbc305def admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:00.528317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b024e9-f6e8-49f5-aab0-f8660d0fa6cd None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:00.533024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ef1ac-19cd-4621-9603-260e2984cf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:00.533768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ef1ac-19cd-4621-9603-260e2984cf7a admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:00.534051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ef1ac-19cd-4621-9603-260e2984cf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:00.534350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ef1ac-19cd-4621-9603-260e2984cf7a admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:00.535052 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ef1ac-19cd-4621-9603-260e2984cf7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:00.536919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ef1ac-19cd-4621-9603-260e2984cf7a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:00.547536 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1086/2185] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:00.552626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ef1ac-19cd-4621-9603-260e2984cf7a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:00.563569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ef1ac-19cd-4621-9603-260e2984cf7a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:00.569495 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c861484-4397-403f-83fb-dd796a8dacf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:00.570166 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861484-4397-403f-83fb-dd796a8dacf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:00.570454 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861484-4397-403f-83fb-dd796a8dacf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:00.570735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861484-4397-403f-83fb-dd796a8dacf1 admin admin] PATH_INFO: `/v3/projects/3f6393499e8c4bf28f13fde6aaea8f78/users/53eb1f17ca054eae8e7d547a057f9b8a/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:00.572753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c861484-4397-403f-83fb-dd796a8dacf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f6393499e8c4bf28f13fde6aaea8f78, user_id=53eb1f17ca054eae8e7d547a057f9b8a, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:00.575427 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c861484-4397-403f-83fb-dd796a8dacf1 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:00.580576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:00.585591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701caab-4fdf-4277-8b09-6739267ff4b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:00.586207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701caab-4fdf-4277-8b09-6739267ff4b2 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:00.586485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701caab-4fdf-4277-8b09-6739267ff4b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:00.586859 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1087/2186] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:00 2022] PUT /identity/v3/projects/3f6393499e8c4bf28f13fde6aaea8f78/users/53eb1f17ca054eae8e7d547a057f9b8a/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:07:00.586975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701caab-4fdf-4277-8b09-6739267ff4b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:00.596117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2701caab-4fdf-4277-8b09-6739267ff4b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:00.597535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2701caab-4fdf-4277-8b09-6739267ff4b2 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:00.605474 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae83b2-3b3b-483f-8fde-d6d3b65e0fd9 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:00.605769 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae83b2-3b3b-483f-8fde-d6d3b65e0fd9 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:00.606054 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae83b2-3b3b-483f-8fde-d6d3b65e0fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:00.607923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1100/2187] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:07:00.638804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-20ae83b2-3b3b-483f-8fde-d6d3b65e0fd9 None None] MFA Rules not processed for user `53eb1f17ca054eae8e7d547a057f9b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:00.640098 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-20ae83b2-3b3b-483f-8fde-d6d3b65e0fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:00.725151 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1088/2188] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:07:00.736658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701caab-4fdf-4277-8b09-6739267ff4b2 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:00.738025 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2701caab-4fdf-4277-8b09-6739267ff4b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:00.796802 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701caab-4fdf-4277-8b09-6739267ff4b2 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:00.869547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e7965-95d5-4754-bc89-8036286c06b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:00.869547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e7965-95d5-4754-bc89-8036286c06b1 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:00.869547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e7965-95d5-4754-bc89-8036286c06b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:00.869547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e7965-95d5-4754-bc89-8036286c06b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:00.883650 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e7965-95d5-4754-bc89-8036286c06b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:00.886135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e7965-95d5-4754-bc89-8036286c06b1 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:00.902569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1101/2189] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:01.151776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae83b2-3b3b-483f-8fde-d6d3b65e0fd9 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:01.173472 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae83b2-3b3b-483f-8fde-d6d3b65e0fd9 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:01.182587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd3cba-15f3-4f94-addb-636b032ac823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:01.182587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd3cba-15f3-4f94-addb-636b032ac823 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:01.182587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd3cba-15f3-4f94-addb-636b032ac823 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:01.182587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd3cba-15f3-4f94-addb-636b032ac823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:01.186423 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-36cd3cba-15f3-4f94-addb-636b032ac823 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:01.305993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd3cba-15f3-4f94-addb-636b032ac823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:01.308042 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd3cba-15f3-4f94-addb-636b032ac823 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:01.322498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1089/2190] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:01.529994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e7965-95d5-4754-bc89-8036286c06b1 service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:01.531599 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-0a8e7965-95d5-4754-bc89-8036286c06b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:01.749212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e7965-95d5-4754-bc89-8036286c06b1 service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:01.758663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457cd29-a1ab-491f-a2c4-c7a538453537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:01.759628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457cd29-a1ab-491f-a2c4-c7a538453537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:01.760239 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457cd29-a1ab-491f-a2c4-c7a538453537 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:01.760756 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457cd29-a1ab-491f-a2c4-c7a538453537 admin admin] PATH_INFO: `/v3/projects/03b62805a4c84aa4a9d57828b98015b6` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:01.764416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457cd29-a1ab-491f-a2c4-c7a538453537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03b62805a4c84aa4a9d57828b98015b6)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:01.768534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457cd29-a1ab-491f-a2c4-c7a538453537 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:01.968366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c457cd29-a1ab-491f-a2c4-c7a538453537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03b62805a4c84aa4a9d57828b98015b6', 'request_id': 'req-c457cd29-a1ab-491f-a2c4-c7a538453537'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:01.972151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:01.981017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c457cd29-a1ab-491f-a2c4-c7a538453537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03b62805a4c84aa4a9d57828b98015b6', 'request_id': 'req-c457cd29-a1ab-491f-a2c4-c7a538453537'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:02.081104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c457cd29-a1ab-491f-a2c4-c7a538453537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03b62805a4c84aa4a9d57828b98015b6', 'request_id': 'req-c457cd29-a1ab-491f-a2c4-c7a538453537'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:02.096862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c457cd29-a1ab-491f-a2c4-c7a538453537 admin admin] The token cache is being invalidate because project 03b62805a4c84aa4a9d57828b98015b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:07:02.097151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c457cd29-a1ab-491f-a2c4-c7a538453537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:02.097815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c457cd29-a1ab-491f-a2c4-c7a538453537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:02.100539 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1102/2191] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:07:01 2022] DELETE /identity/v3/projects/03b62805a4c84aa4a9d57828b98015b6 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:02.105598 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58e8a1-728c-4f42-9361-dff86fd099ef None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:02.106580 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58e8a1-728c-4f42-9361-dff86fd099ef None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:02.107328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58e8a1-728c-4f42-9361-dff86fd099ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:02.133184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-8f58e8a1-728c-4f42-9361-dff86fd099ef None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:02.134627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8f58e8a1-728c-4f42-9361-dff86fd099ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:02.150349 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:02.212040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1090/2192] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:02.215081 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc70a4-75ba-4d87-99d7-2d317d1c01ff None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:02.215081 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc70a4-75ba-4d87-99d7-2d317d1c01ff None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:02.215370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc70a4-75ba-4d87-99d7-2d317d1c01ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:02.241015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-43bc70a4-75ba-4d87-99d7-2d317d1c01ff None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:02.242282 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-43bc70a4-75ba-4d87-99d7-2d317d1c01ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:02.260860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1103/2193] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:02.270314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58e8a1-728c-4f42-9361-dff86fd099ef None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:02.289086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58e8a1-728c-4f42-9361-dff86fd099ef None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:02.298742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254356e-f97d-4c32-ad9d-f060d5bb08a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:02.299390 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254356e-f97d-4c32-ad9d-f060d5bb08a3 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:02.299766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254356e-f97d-4c32-ad9d-f060d5bb08a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:02.300281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254356e-f97d-4c32-ad9d-f060d5bb08a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:02.301406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254356e-f97d-4c32-ad9d-f060d5bb08a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:07:02.301766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254356e-f97d-4c32-ad9d-f060d5bb08a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:02.303666 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254356e-f97d-4c32-ad9d-f060d5bb08a3 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:02.324307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc70a4-75ba-4d87-99d7-2d317d1c01ff None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:02.330070 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1091/2194] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:07:02.336397 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc70a4-75ba-4d87-99d7-2d317d1c01ff None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:02.343211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bd7c1-fab8-43aa-a556-0355fa45066d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:02.344340 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bd7c1-fab8-43aa-a556-0355fa45066d admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:02.344759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bd7c1-fab8-43aa-a556-0355fa45066d admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:02.345331 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bd7c1-fab8-43aa-a556-0355fa45066d admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:02.346015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20bd7c1-fab8-43aa-a556-0355fa45066d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:02.348429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20bd7c1-fab8-43aa-a556-0355fa45066d admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:02.560150 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1104/2195] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:07:02 2022] POST /identity/v3/projects => generated 389 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:02.564506 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254356e-f97d-4c32-ad9d-f060d5bb08a3 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:02.576341 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254356e-f97d-4c32-ad9d-f060d5bb08a3 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:02.651275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb3f1d-2972-47d2-9161-35bf0be9752f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:02.651942 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb3f1d-2972-47d2-9161-35bf0be9752f admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:02.652227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb3f1d-2972-47d2-9161-35bf0be9752f admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:02.652607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb3f1d-2972-47d2-9161-35bf0be9752f admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:02.653195 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb3f1d-2972-47d2-9161-35bf0be9752f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:02.655135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb3f1d-2972-47d2-9161-35bf0be9752f admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:02.816025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1092/2196] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:07:02 2022] POST /identity/v3/users => generated 295 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:02.820057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bd7c1-fab8-43aa-a556-0355fa45066d admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:02.831496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bd7c1-fab8-43aa-a556-0355fa45066d admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:02.837038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad32463-4c2b-4799-ac06-265511b86411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:02.837690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad32463-4c2b-4799-ac06-265511b86411 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:02.842669 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad32463-4c2b-4799-ac06-265511b86411 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:02.842669 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad32463-4c2b-4799-ac06-265511b86411 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:02.842669 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad32463-4c2b-4799-ac06-265511b86411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:02.842669 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad32463-4c2b-4799-ac06-265511b86411 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:02.852075 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1105/2197] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:02.855950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb3f1d-2972-47d2-9161-35bf0be9752f admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:02.869152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb3f1d-2972-47d2-9161-35bf0be9752f admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:02.875598 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617df95f-4ea0-41e3-999b-60dcba56da5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:02.877887 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617df95f-4ea0-41e3-999b-60dcba56da5c admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:02.878773 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617df95f-4ea0-41e3-999b-60dcba56da5c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:02.879057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617df95f-4ea0-41e3-999b-60dcba56da5c admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:02.891487 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617df95f-4ea0-41e3-999b-60dcba56da5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:02.891487 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617df95f-4ea0-41e3-999b-60dcba56da5c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:02.908365 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1093/2198] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:02.913008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad32463-4c2b-4799-ac06-265511b86411 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:02.926431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad32463-4c2b-4799-ac06-265511b86411 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:02.938949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b9623-bc18-443c-a709-d8078b47f9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:02.939829 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b9623-bc18-443c-a709-d8078b47f9b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:02.940149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b9623-bc18-443c-a709-d8078b47f9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:02.940482 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b9623-bc18-443c-a709-d8078b47f9b8 admin admin] PATH_INFO: `/v3/projects/5cc2bf83118e4b22a3fc0678236d2018/users/cc6095af50fa473cbdede2213b0000b2/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:02.960311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b9623-bc18-443c-a709-d8078b47f9b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc2bf83118e4b22a3fc0678236d2018, user_id=cc6095af50fa473cbdede2213b0000b2, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:02.964053 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b9623-bc18-443c-a709-d8078b47f9b8 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:03.084192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1106/2199] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:02 2022] PUT /identity/v3/projects/5cc2bf83118e4b22a3fc0678236d2018/users/cc6095af50fa473cbdede2213b0000b2/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:07:03.084947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3322f0-b881-47e8-9063-66e3afdff555 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:03.085290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3322f0-b881-47e8-9063-66e3afdff555 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:03.085594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3322f0-b881-47e8-9063-66e3afdff555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:03.135532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ff3322f0-b881-47e8-9063-66e3afdff555 None None] MFA Rules not processed for user `cc6095af50fa473cbdede2213b0000b2`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:03.137650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ff3322f0-b881-47e8-9063-66e3afdff555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:03.231106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1094/2200] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:07:03.243064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b9623-bc18-443c-a709-d8078b47f9b8 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:03.244977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-fb7b9623-bc18-443c-a709-d8078b47f9b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:03.310620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b9623-bc18-443c-a709-d8078b47f9b8 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:03.314484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77831e7-6811-49b5-899f-79e39a9cb8cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:03.315209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77831e7-6811-49b5-899f-79e39a9cb8cf service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:03.315580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77831e7-6811-49b5-899f-79e39a9cb8cf service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:03.315900 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77831e7-6811-49b5-899f-79e39a9cb8cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:03.326638 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77831e7-6811-49b5-899f-79e39a9cb8cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:03.328201 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77831e7-6811-49b5-899f-79e39a9cb8cf service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:03.339079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1107/2201] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:07:13.993121 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=59,keystone:DELETE=2,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:14.798352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:22.395577 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3322f0-b881-47e8-9063-66e3afdff555 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:22.397136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ff3322f0-b881-47e8-9063-66e3afdff555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:22.466143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3322f0-b881-47e8-9063-66e3afdff555 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:22.471484 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3bf0-f704-42f7-ae8c-16cad8e2b15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:22.471484 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3bf0-f704-42f7-ae8c-16cad8e2b15e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:22.472085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3bf0-f704-42f7-ae8c-16cad8e2b15e admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:22.472085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3bf0-f704-42f7-ae8c-16cad8e2b15e admin admin] PATH_INFO: `/v3/users/16db51d64cd54ac5a6090b76729783cb` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:22.474304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f3bf0-f704-42f7-ae8c-16cad8e2b15e admin admin] RBAC: Authorizing `identity:delete_user(user_id=16db51d64cd54ac5a6090b76729783cb)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:22.476292 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f3bf0-f704-42f7-ae8c-16cad8e2b15e admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:22.701891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-c79f3bf0-f704-42f7-ae8c-16cad8e2b15e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16db51d64cd54ac5a6090b76729783cb'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:22.702731 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-c79f3bf0-f704-42f7-ae8c-16cad8e2b15e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16db51d64cd54ac5a6090b76729783cb'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:22.703554 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-c79f3bf0-f704-42f7-ae8c-16cad8e2b15e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16db51d64cd54ac5a6090b76729783cb'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:22.713422 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-c79f3bf0-f704-42f7-ae8c-16cad8e2b15e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16db51d64cd54ac5a6090b76729783cb'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:22.729643 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-c79f3bf0-f704-42f7-ae8c-16cad8e2b15e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16db51d64cd54ac5a6090b76729783cb'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:22.837480 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1095/2202] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:07:22 2022] DELETE /identity/v3/users/16db51d64cd54ac5a6090b76729783cb => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:22.842201 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ba80d-c503-49e8-b084-a4e1d6787ab7 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:22.842721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ba80d-c503-49e8-b084-a4e1d6787ab7 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:22.842819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ba80d-c503-49e8-b084-a4e1d6787ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:22.866249 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-4d7ba80d-c503-49e8-b084-a4e1d6787ab7 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:22.867487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4d7ba80d-c503-49e8-b084-a4e1d6787ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:22.930438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1108/2203] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:22.940658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3bf0-f704-42f7-ae8c-16cad8e2b15e admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:22.942185 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c79f3bf0-f704-42f7-ae8c-16cad8e2b15e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:23.001594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3bf0-f704-42f7-ae8c-16cad8e2b15e admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:23.005480 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b0726-3574-441e-ad48-2ef581109e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:23.006085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b0726-3574-441e-ad48-2ef581109e87 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:23.006362 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b0726-3574-441e-ad48-2ef581109e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:23.006636 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b0726-3574-441e-ad48-2ef581109e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:23.015738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b0726-3574-441e-ad48-2ef581109e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:23.017152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b0726-3574-441e-ad48-2ef581109e87 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:23.027263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1096/2204] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:23.481786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ba80d-c503-49e8-b084-a4e1d6787ab7 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:23.483381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4d7ba80d-c503-49e8-b084-a4e1d6787ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:23.496957 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ba80d-c503-49e8-b084-a4e1d6787ab7 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:23.501707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:23.502360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:23.502643 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:23.503065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] PATH_INFO: `/v3/projects/8dcc2cf3cf2f49348683645556b0c2d7` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:23.504788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dcc2cf3cf2f49348683645556b0c2d7)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:23.506730 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:23.615849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dcc2cf3cf2f49348683645556b0c2d7', 'request_id': 'req-f7165fc5-5df6-412f-becf-28c9a3d839b6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:23.621819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dcc2cf3cf2f49348683645556b0c2d7', 'request_id': 'req-f7165fc5-5df6-412f-becf-28c9a3d839b6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:23.710574 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dcc2cf3cf2f49348683645556b0c2d7', 'request_id': 'req-f7165fc5-5df6-412f-becf-28c9a3d839b6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:23.722709 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] The token cache is being invalidate because project 8dcc2cf3cf2f49348683645556b0c2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:07:23.723149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:23.723818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:23.725348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1109/2205] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:07:23 2022] DELETE /identity/v3/projects/8dcc2cf3cf2f49348683645556b0c2d7 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:23.730096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b0726-3574-441e-ad48-2ef581109e87 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:23.740695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b0726-3574-441e-ad48-2ef581109e87 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:23.761503 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:23.802801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b0452-93b9-45c8-bd7c-31fd116301b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:23.803442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b0452-93b9-45c8-bd7c-31fd116301b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:23.803895 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b0452-93b9-45c8-bd7c-31fd116301b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:23.804230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b0452-93b9-45c8-bd7c-31fd116301b4 admin admin] PATH_INFO: `/v3/users/53eb1f17ca054eae8e7d547a057f9b8a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:23.805425 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b0452-93b9-45c8-bd7c-31fd116301b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53eb1f17ca054eae8e7d547a057f9b8a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:23.807691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b0452-93b9-45c8-bd7c-31fd116301b4 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:23.997815 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2c1b0452-93b9-45c8-bd7c-31fd116301b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53eb1f17ca054eae8e7d547a057f9b8a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:23.998476 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2c1b0452-93b9-45c8-bd7c-31fd116301b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53eb1f17ca054eae8e7d547a057f9b8a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:23.999079 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2c1b0452-93b9-45c8-bd7c-31fd116301b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53eb1f17ca054eae8e7d547a057f9b8a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:24.006578 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2c1b0452-93b9-45c8-bd7c-31fd116301b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53eb1f17ca054eae8e7d547a057f9b8a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:24.018441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2c1b0452-93b9-45c8-bd7c-31fd116301b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53eb1f17ca054eae8e7d547a057f9b8a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:24.128329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1097/2206] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:07:23 2022] DELETE /identity/v3/users/53eb1f17ca054eae8e7d547a057f9b8a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:24.563173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:24.564793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:24.579974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7165fc5-5df6-412f-becf-28c9a3d839b6 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:24.665490 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:24.666142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:24.666565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:24.666818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6 admin admin] PATH_INFO: `/v3/projects/3f6393499e8c4bf28f13fde6aaea8f78` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:24.667903 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f6393499e8c4bf28f13fde6aaea8f78)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:24.669884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:24.788113 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:24.807804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f6393499e8c4bf28f13fde6aaea8f78', 'request_id': 'req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:24.813362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f6393499e8c4bf28f13fde6aaea8f78', 'request_id': 'req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:24.916321 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f6393499e8c4bf28f13fde6aaea8f78', 'request_id': 'req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:24.929084 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6 admin admin] The token cache is being invalidate because project 3f6393499e8c4bf28f13fde6aaea8f78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:07:24.930818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:24.931605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-df8cb893-1e70-4aa2-b8c9-b69d0be414f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:24.933221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1110/2207] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:07:24 2022] DELETE /identity/v3/projects/3f6393499e8c4bf28f13fde6aaea8f78 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:24.939092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9118ce3-e983-410d-b01b-4c2c1a145349 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:24.939609 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9118ce3-e983-410d-b01b-4c2c1a145349 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:24.939718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9118ce3-e983-410d-b01b-4c2c1a145349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:24.959854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-c9118ce3-e983-410d-b01b-4c2c1a145349 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:24.961189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c9118ce3-e983-410d-b01b-4c2c1a145349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:25.027590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1098/2208] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:25.034909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc2011-f854-4d43-84d8-cf21357bd008 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:25.036100 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc2011-f854-4d43-84d8-cf21357bd008 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:25.036380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc2011-f854-4d43-84d8-cf21357bd008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:25.058883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-dedc2011-f854-4d43-84d8-cf21357bd008 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:25.060733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-dedc2011-f854-4d43-84d8-cf21357bd008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:25.071411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1111/2209] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:25.076844 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9118ce3-e983-410d-b01b-4c2c1a145349 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:25.089844 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9118ce3-e983-410d-b01b-4c2c1a145349 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:25.094870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f7ddf-a590-4079-83ea-58416e9ded0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:25.095731 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f7ddf-a590-4079-83ea-58416e9ded0c admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:25.096305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f7ddf-a590-4079-83ea-58416e9ded0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:25.099790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f7ddf-a590-4079-83ea-58416e9ded0c admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:25.099790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f7ddf-a590-4079-83ea-58416e9ded0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:07:25.099790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f7ddf-a590-4079-83ea-58416e9ded0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:25.100610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f7ddf-a590-4079-83ea-58416e9ded0c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:25.113442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1099/2210] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:07:25.117933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc2011-f854-4d43-84d8-cf21357bd008 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:25.128202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc2011-f854-4d43-84d8-cf21357bd008 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:25.134011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a499138-5502-4563-9dc6-de568f238fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:25.134675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a499138-5502-4563-9dc6-de568f238fba admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:25.134955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a499138-5502-4563-9dc6-de568f238fba admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:25.135228 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a499138-5502-4563-9dc6-de568f238fba admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:25.135951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a499138-5502-4563-9dc6-de568f238fba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:25.137903 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a499138-5502-4563-9dc6-de568f238fba admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:25.245013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1112/2211] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:07:25 2022] POST /identity/v3/projects => generated 401 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:25.249175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f7ddf-a590-4079-83ea-58416e9ded0c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:25.259890 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f7ddf-a590-4079-83ea-58416e9ded0c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:25.292421 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:25.328020 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bd0df-b973-45e3-94ab-703b9f32e216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:25.328640 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bd0df-b973-45e3-94ab-703b9f32e216 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:25.328922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bd0df-b973-45e3-94ab-703b9f32e216 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:25.329198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bd0df-b973-45e3-94ab-703b9f32e216 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:25.329917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bd0df-b973-45e3-94ab-703b9f32e216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:25.332448 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184bd0df-b973-45e3-94ab-703b9f32e216 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:25.596397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1100/2212] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:07:25 2022] POST /identity/v3/users => generated 301 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:07:25.600913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a499138-5502-4563-9dc6-de568f238fba admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:25.611281 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a499138-5502-4563-9dc6-de568f238fba admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:25.615918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14388c-1765-4a05-97d0-b07a627e1473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:25.616524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14388c-1765-4a05-97d0-b07a627e1473 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:25.616820 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14388c-1765-4a05-97d0-b07a627e1473 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:25.617073 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14388c-1765-4a05-97d0-b07a627e1473 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:25.618334 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14388c-1765-4a05-97d0-b07a627e1473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:25.619970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14388c-1765-4a05-97d0-b07a627e1473 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:25.631411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1113/2213] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:25.636217 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bd0df-b973-45e3-94ab-703b9f32e216 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:25.649685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bd0df-b973-45e3-94ab-703b9f32e216 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:25.659698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b040bf8-1891-46e0-8efe-217113216d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:25.662663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b040bf8-1891-46e0-8efe-217113216d07 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:25.662958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b040bf8-1891-46e0-8efe-217113216d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:25.663285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b040bf8-1891-46e0-8efe-217113216d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:25.664052 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b040bf8-1891-46e0-8efe-217113216d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:25.666126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b040bf8-1891-46e0-8efe-217113216d07 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:25.677358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1101/2214] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:25.681577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14388c-1765-4a05-97d0-b07a627e1473 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:25.693062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14388c-1765-4a05-97d0-b07a627e1473 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:25.699368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b241dfa-3e2d-41e7-abff-53b6ee9c7d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:25.700166 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b241dfa-3e2d-41e7-abff-53b6ee9c7d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:25.700537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b241dfa-3e2d-41e7-abff-53b6ee9c7d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:25.701011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b241dfa-3e2d-41e7-abff-53b6ee9c7d05 admin admin] PATH_INFO: `/v3/projects/30cfba60d2fe4fdb8d208be6e8726475/users/a124a80fc9d14f5bbcf63370887b08e1/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:25.724708 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b241dfa-3e2d-41e7-abff-53b6ee9c7d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30cfba60d2fe4fdb8d208be6e8726475, user_id=a124a80fc9d14f5bbcf63370887b08e1, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:25.727370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b241dfa-3e2d-41e7-abff-53b6ee9c7d05 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:25.820184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1114/2215] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:25 2022] PUT /identity/v3/projects/30cfba60d2fe4fdb8d208be6e8726475/users/a124a80fc9d14f5bbcf63370887b08e1/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:25.822314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f176ec-1235-40c3-a4b1-1f3da20c9343 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:25.822667 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f176ec-1235-40c3-a4b1-1f3da20c9343 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:25.823009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f176ec-1235-40c3-a4b1-1f3da20c9343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:25.856006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-f6f176ec-1235-40c3-a4b1-1f3da20c9343 None None] MFA Rules not processed for user `a124a80fc9d14f5bbcf63370887b08e1`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:25.857609 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f6f176ec-1235-40c3-a4b1-1f3da20c9343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:25.971626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1102/2216] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:25.976548 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b241dfa-3e2d-41e7-abff-53b6ee9c7d05 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:26.070106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b241dfa-3e2d-41e7-abff-53b6ee9c7d05 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:26.075994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cca748-610c-4c37-9304-1de7e0c76ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:26.076676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cca748-610c-4c37-9304-1de7e0c76ddb admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:26.076964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cca748-610c-4c37-9304-1de7e0c76ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:26.077320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cca748-610c-4c37-9304-1de7e0c76ddb admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:26.077947 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cca748-610c-4c37-9304-1de7e0c76ddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:26.079923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cca748-610c-4c37-9304-1de7e0c76ddb admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:26.203981 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1115/2217] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:07:25 2022] POST /identity/v3/projects => generated 395 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:26.208722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f176ec-1235-40c3-a4b1-1f3da20c9343 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:26.220545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f176ec-1235-40c3-a4b1-1f3da20c9343 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:26.298207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81109ce4-be88-4e04-8310-a4af7f07472c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:26.298924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81109ce4-be88-4e04-8310-a4af7f07472c admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:26.299221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81109ce4-be88-4e04-8310-a4af7f07472c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:26.299679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81109ce4-be88-4e04-8310-a4af7f07472c admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:26.300364 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81109ce4-be88-4e04-8310-a4af7f07472c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:26.303647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81109ce4-be88-4e04-8310-a4af7f07472c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:26.470457 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1103/2218] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:07:26 2022] POST /identity/v3/users => generated 304 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:26.473620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cca748-610c-4c37-9304-1de7e0c76ddb admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:26.485224 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cca748-610c-4c37-9304-1de7e0c76ddb admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:26.491091 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b723756-b71d-4ae7-99a9-1400f30fbc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:26.492056 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b723756-b71d-4ae7-99a9-1400f30fbc62 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:26.492229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b723756-b71d-4ae7-99a9-1400f30fbc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:26.492565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b723756-b71d-4ae7-99a9-1400f30fbc62 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:26.493343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b723756-b71d-4ae7-99a9-1400f30fbc62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:26.494989 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b723756-b71d-4ae7-99a9-1400f30fbc62 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:26.507688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1116/2219] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:26.511955 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81109ce4-be88-4e04-8310-a4af7f07472c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:26.530458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81109ce4-be88-4e04-8310-a4af7f07472c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:26.538666 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab245d-77a6-4d48-be76-bf056bddc142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:26.539650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab245d-77a6-4d48-be76-bf056bddc142 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:26.540123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab245d-77a6-4d48-be76-bf056bddc142 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:26.540591 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab245d-77a6-4d48-be76-bf056bddc142 admin admin] PATH_INFO: `/v3/projects/8187a4395efb4338ae1fd75cbb5ce775/users/8a96896672244b4c91492535286c42bc/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:26.567723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab245d-77a6-4d48-be76-bf056bddc142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8187a4395efb4338ae1fd75cbb5ce775, user_id=8a96896672244b4c91492535286c42bc, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:26.570470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab245d-77a6-4d48-be76-bf056bddc142 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:26.743940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1104/2220] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:26 2022] PUT /identity/v3/projects/8187a4395efb4338ae1fd75cbb5ce775/users/8a96896672244b4c91492535286c42bc/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:26.747044 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b723756-b71d-4ae7-99a9-1400f30fbc62 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:26.759128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b723756-b71d-4ae7-99a9-1400f30fbc62 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:26.791845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:26.835004 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ac2cf-2a33-4ec7-a94f-5268324e0edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:26.835712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ac2cf-2a33-4ec7-a94f-5268324e0edf admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:26.835991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ac2cf-2a33-4ec7-a94f-5268324e0edf admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:26.836259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ac2cf-2a33-4ec7-a94f-5268324e0edf admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:26.836955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ac2cf-2a33-4ec7-a94f-5268324e0edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:26.838531 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ac2cf-2a33-4ec7-a94f-5268324e0edf admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:26.849845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1117/2221] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:26.854316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab245d-77a6-4d48-be76-bf056bddc142 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:26.860380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:26.866843 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab245d-77a6-4d48-be76-bf056bddc142 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:26.872449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f607691-6997-492c-8553-a28d10b87fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:26.873501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f607691-6997-492c-8553-a28d10b87fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:26.873865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f607691-6997-492c-8553-a28d10b87fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:26.874491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f607691-6997-492c-8553-a28d10b87fcb admin admin] PATH_INFO: `/v3/projects/8187a4395efb4338ae1fd75cbb5ce775/users/8a96896672244b4c91492535286c42bc/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:26.878247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f607691-6997-492c-8553-a28d10b87fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8187a4395efb4338ae1fd75cbb5ce775, user_id=8a96896672244b4c91492535286c42bc, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:26.883122 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f607691-6997-492c-8553-a28d10b87fcb admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:26.898733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1105/2222] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:26 2022] PUT /identity/v3/projects/8187a4395efb4338ae1fd75cbb5ce775/users/8a96896672244b4c91492535286c42bc/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:26.913606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34557c2-d3ba-4eca-b524-b554fe64ae6d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:26.914038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34557c2-d3ba-4eca-b524-b554fe64ae6d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:26.914755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34557c2-d3ba-4eca-b524-b554fe64ae6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:26.952505 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-f34557c2-d3ba-4eca-b524-b554fe64ae6d None None] MFA Rules not processed for user `8a96896672244b4c91492535286c42bc`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:26.953812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f34557c2-d3ba-4eca-b524-b554fe64ae6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:27.034543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1118/2223] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:27.150708 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f607691-6997-492c-8553-a28d10b87fcb admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:27.152734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-5f607691-6997-492c-8553-a28d10b87fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:27.214770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f607691-6997-492c-8553-a28d10b87fcb admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:27.218409 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec19dc-4856-4f24-a54f-18e5956f989a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:27.219024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec19dc-4856-4f24-a54f-18e5956f989a service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:27.219303 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec19dc-4856-4f24-a54f-18e5956f989a service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:27.219617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec19dc-4856-4f24-a54f-18e5956f989a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:27.229727 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec19dc-4856-4f24-a54f-18e5956f989a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:27.231206 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec19dc-4856-4f24-a54f-18e5956f989a service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:27.248776 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1106/2224] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:27.770547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34557c2-d3ba-4eca-b524-b554fe64ae6d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:27.772312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f34557c2-d3ba-4eca-b524-b554fe64ae6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:27.789100 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34557c2-d3ba-4eca-b524-b554fe64ae6d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:27.792729 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a1303-b5f7-4b16-a29a-df4f8f65dde5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:27.797467 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a1303-b5f7-4b16-a29a-df4f8f65dde5 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:27.797467 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a1303-b5f7-4b16-a29a-df4f8f65dde5 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:27.797467 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a1303-b5f7-4b16-a29a-df4f8f65dde5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:27.858282 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a1303-b5f7-4b16-a29a-df4f8f65dde5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:27.859809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a1303-b5f7-4b16-a29a-df4f8f65dde5 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:27.870023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1119/2225] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:28.428540 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec19dc-4856-4f24-a54f-18e5956f989a service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:28.430398 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-60ec19dc-4856-4f24-a54f-18e5956f989a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:28.521183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec19dc-4856-4f24-a54f-18e5956f989a service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:28.526455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ca165-142a-41f3-bea4-d0ab7a0c4349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:28.527119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ca165-142a-41f3-bea4-d0ab7a0c4349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:28.527430 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ca165-142a-41f3-bea4-d0ab7a0c4349 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:28.527734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ca165-142a-41f3-bea4-d0ab7a0c4349 admin admin] PATH_INFO: `/v3/users/c718453b1b2b428dac800007f031de6b` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:28.529072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ca165-142a-41f3-bea4-d0ab7a0c4349 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c718453b1b2b428dac800007f031de6b)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:28.531315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ca165-142a-41f3-bea4-d0ab7a0c4349 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:28.833208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-0e6ca165-142a-41f3-bea4-d0ab7a0c4349 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c718453b1b2b428dac800007f031de6b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:28.834100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-0e6ca165-142a-41f3-bea4-d0ab7a0c4349 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c718453b1b2b428dac800007f031de6b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:28.834792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-0e6ca165-142a-41f3-bea4-d0ab7a0c4349 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c718453b1b2b428dac800007f031de6b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:28.843373 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-0e6ca165-142a-41f3-bea4-d0ab7a0c4349 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c718453b1b2b428dac800007f031de6b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:28.857909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-0e6ca165-142a-41f3-bea4-d0ab7a0c4349 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c718453b1b2b428dac800007f031de6b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:28.931208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1107/2226] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:07:28 2022] DELETE /identity/v3/users/c718453b1b2b428dac800007f031de6b => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:28.935946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82269e-6979-4cd2-9040-1e0bcdddad1d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:28.936449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82269e-6979-4cd2-9040-1e0bcdddad1d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:28.936563 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82269e-6979-4cd2-9040-1e0bcdddad1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:28.958610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-3f82269e-6979-4cd2-9040-1e0bcdddad1d None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:28.959886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3f82269e-6979-4cd2-9040-1e0bcdddad1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:29.003924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:29.024683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1120/2227] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:29.036025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ca165-142a-41f3-bea4-d0ab7a0c4349 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:29.037503 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0e6ca165-142a-41f3-bea4-d0ab7a0c4349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:29.051794 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:29.099780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ca165-142a-41f3-bea4-d0ab7a0c4349 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:29.103853 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e574b05-4b4f-4ab4-994d-dabde03de200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:29.104586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e574b05-4b4f-4ab4-994d-dabde03de200 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:29.104871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e574b05-4b4f-4ab4-994d-dabde03de200 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:29.105259 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e574b05-4b4f-4ab4-994d-dabde03de200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:29.115302 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e574b05-4b4f-4ab4-994d-dabde03de200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:29.117264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e574b05-4b4f-4ab4-994d-dabde03de200 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:29.127224 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1108/2228] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:29.664275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82269e-6979-4cd2-9040-1e0bcdddad1d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:29.666039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3f82269e-6979-4cd2-9040-1e0bcdddad1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:29.679511 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82269e-6979-4cd2-9040-1e0bcdddad1d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:29.685947 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd278c02-434a-4b7c-a521-9e93a00bf84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:29.686962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd278c02-434a-4b7c-a521-9e93a00bf84e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:29.687500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd278c02-434a-4b7c-a521-9e93a00bf84e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:29.687844 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd278c02-434a-4b7c-a521-9e93a00bf84e admin admin] PATH_INFO: `/v3/projects/06f44e6d3d164242a9107933b6144e2f` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:29.689242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd278c02-434a-4b7c-a521-9e93a00bf84e admin admin] RBAC: Authorizing `identity:delete_project(project_id=06f44e6d3d164242a9107933b6144e2f)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:29.699059 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd278c02-434a-4b7c-a521-9e93a00bf84e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:29.878531 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-dd278c02-434a-4b7c-a521-9e93a00bf84e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06f44e6d3d164242a9107933b6144e2f', 'request_id': 'req-dd278c02-434a-4b7c-a521-9e93a00bf84e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:29.886615 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-dd278c02-434a-4b7c-a521-9e93a00bf84e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06f44e6d3d164242a9107933b6144e2f', 'request_id': 'req-dd278c02-434a-4b7c-a521-9e93a00bf84e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:29.994868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-dd278c02-434a-4b7c-a521-9e93a00bf84e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06f44e6d3d164242a9107933b6144e2f', 'request_id': 'req-dd278c02-434a-4b7c-a521-9e93a00bf84e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:30.010956 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-dd278c02-434a-4b7c-a521-9e93a00bf84e admin admin] The token cache is being invalidate because project 06f44e6d3d164242a9107933b6144e2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:07:30.011348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-dd278c02-434a-4b7c-a521-9e93a00bf84e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:30.012360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-dd278c02-434a-4b7c-a521-9e93a00bf84e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:30.014816 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1121/2229] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:07:29 2022] DELETE /identity/v3/projects/06f44e6d3d164242a9107933b6144e2f => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:30.020294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaca284-dc4e-466e-8daf-5c2a442b6b92 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:30.020911 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaca284-dc4e-466e-8daf-5c2a442b6b92 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:30.021090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaca284-dc4e-466e-8daf-5c2a442b6b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:30.064075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-fbaca284-dc4e-466e-8daf-5c2a442b6b92 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:30.065396 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-fbaca284-dc4e-466e-8daf-5c2a442b6b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:30.133532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1109/2230] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:30.142316 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2f91f-2d65-462f-9b0b-46f6e73e528f None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:30.142603 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2f91f-2d65-462f-9b0b-46f6e73e528f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:30.142912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2f91f-2d65-462f-9b0b-46f6e73e528f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:30.163887 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-4ab2f91f-2d65-462f-9b0b-46f6e73e528f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:30.165095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4ab2f91f-2d65-462f-9b0b-46f6e73e528f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:30.173010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1122/2231] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:30.179173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaca284-dc4e-466e-8daf-5c2a442b6b92 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:30.188909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaca284-dc4e-466e-8daf-5c2a442b6b92 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:30.193536 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf5852-b4b3-4bdc-81a6-c699469c86bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:30.194198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf5852-b4b3-4bdc-81a6-c699469c86bd admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:30.194492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf5852-b4b3-4bdc-81a6-c699469c86bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:30.194855 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf5852-b4b3-4bdc-81a6-c699469c86bd admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:30.195609 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf5852-b4b3-4bdc-81a6-c699469c86bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:07:30.195928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf5852-b4b3-4bdc-81a6-c699469c86bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:30.197550 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf5852-b4b3-4bdc-81a6-c699469c86bd admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:30.212796 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1110/2232] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:07:30.220748 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2f91f-2d65-462f-9b0b-46f6e73e528f None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:30.231020 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2f91f-2d65-462f-9b0b-46f6e73e528f None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:30.236171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54310a96-c8ac-47f7-9e5d-4886586d8e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:30.236774 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54310a96-c8ac-47f7-9e5d-4886586d8e08 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:30.237050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54310a96-c8ac-47f7-9e5d-4886586d8e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:30.237554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54310a96-c8ac-47f7-9e5d-4886586d8e08 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:30.238228 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54310a96-c8ac-47f7-9e5d-4886586d8e08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:30.240672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54310a96-c8ac-47f7-9e5d-4886586d8e08 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:30.358813 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1123/2233] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:07:30 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:30.363617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf5852-b4b3-4bdc-81a6-c699469c86bd admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:30.380730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf5852-b4b3-4bdc-81a6-c699469c86bd admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:30.444132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d63a9-0f8a-46e7-b2f8-9902d365f0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:30.444808 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d63a9-0f8a-46e7-b2f8-9902d365f0df admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:30.445097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d63a9-0f8a-46e7-b2f8-9902d365f0df admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:30.445421 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d63a9-0f8a-46e7-b2f8-9902d365f0df admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:30.446239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d63a9-0f8a-46e7-b2f8-9902d365f0df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:30.448402 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d63a9-0f8a-46e7-b2f8-9902d365f0df admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:30.558617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1111/2234] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:07:30 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:07:30.564079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54310a96-c8ac-47f7-9e5d-4886586d8e08 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:30.575233 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54310a96-c8ac-47f7-9e5d-4886586d8e08 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:30.582134 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483fcb21-9d2a-4a82-977a-e8a6918e6a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:30.583046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fcb21-9d2a-4a82-977a-e8a6918e6a58 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:30.583993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fcb21-9d2a-4a82-977a-e8a6918e6a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:30.584578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fcb21-9d2a-4a82-977a-e8a6918e6a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:30.585419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483fcb21-9d2a-4a82-977a-e8a6918e6a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:30.587695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483fcb21-9d2a-4a82-977a-e8a6918e6a58 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:30.601336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1124/2235] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:07:30.606306 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d63a9-0f8a-46e7-b2f8-9902d365f0df admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:30.618991 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d63a9-0f8a-46e7-b2f8-9902d365f0df admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:30.626635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb0bf7-6d7d-45d6-940f-6974f6caeba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:30.627281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb0bf7-6d7d-45d6-940f-6974f6caeba4 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:30.627616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb0bf7-6d7d-45d6-940f-6974f6caeba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:30.627899 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb0bf7-6d7d-45d6-940f-6974f6caeba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:30.628792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb0bf7-6d7d-45d6-940f-6974f6caeba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:30.634116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb0bf7-6d7d-45d6-940f-6974f6caeba4 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:30.657405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1112/2236] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:07:30.661031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483fcb21-9d2a-4a82-977a-e8a6918e6a58 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:30.676815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483fcb21-9d2a-4a82-977a-e8a6918e6a58 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:30.688508 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb9cf6-f23d-4359-8b8d-cfb7b50c8da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:30.689303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb9cf6-f23d-4359-8b8d-cfb7b50c8da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:30.689869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb9cf6-f23d-4359-8b8d-cfb7b50c8da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:30.690067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb9cf6-f23d-4359-8b8d-cfb7b50c8da2 admin admin] PATH_INFO: `/v3/projects/ad6d7ffaa7104eb68ce544d95dd4605e/users/02da7ee5801e424aac67a6b40ef46932/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:30.711149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb9cf6-f23d-4359-8b8d-cfb7b50c8da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6d7ffaa7104eb68ce544d95dd4605e, user_id=02da7ee5801e424aac67a6b40ef46932, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:30.714125 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb9cf6-f23d-4359-8b8d-cfb7b50c8da2 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:30.829399 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1125/2237] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:30 2022] PUT /identity/v3/projects/ad6d7ffaa7104eb68ce544d95dd4605e/users/02da7ee5801e424aac67a6b40ef46932/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:30.841708 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19955129-9cd6-4700-bf84-ad73b9f7a13e None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:30.841991 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19955129-9cd6-4700-bf84-ad73b9f7a13e None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:30.842266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19955129-9cd6-4700-bf84-ad73b9f7a13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:30.870608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-19955129-9cd6-4700-bf84-ad73b9f7a13e None None] MFA Rules not processed for user `02da7ee5801e424aac67a6b40ef46932`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:30.872014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-19955129-9cd6-4700-bf84-ad73b9f7a13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:30.880954 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:30.941389 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1113/2238] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:30.956573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb9cf6-f23d-4359-8b8d-cfb7b50c8da2 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:30.958208 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-cdeb9cf6-f23d-4359-8b8d-cfb7b50c8da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:31.020172 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb9cf6-f23d-4359-8b8d-cfb7b50c8da2 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:31.024579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879755b6-5be4-4d9a-99ec-be34dce4138d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:31.025206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879755b6-5be4-4d9a-99ec-be34dce4138d service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:31.025580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879755b6-5be4-4d9a-99ec-be34dce4138d service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:31.025934 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879755b6-5be4-4d9a-99ec-be34dce4138d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:31.041221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879755b6-5be4-4d9a-99ec-be34dce4138d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:31.042386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879755b6-5be4-4d9a-99ec-be34dce4138d service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:31.051960 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1126/2239] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:32.428966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19955129-9cd6-4700-bf84-ad73b9f7a13e None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:32.431438 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-19955129-9cd6-4700-bf84-ad73b9f7a13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:32.522224 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19955129-9cd6-4700-bf84-ad73b9f7a13e None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:32.527135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2d593-1d42-441d-8e62-1913f25f652a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:32.527820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2d593-1d42-441d-8e62-1913f25f652a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:32.528104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2d593-1d42-441d-8e62-1913f25f652a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:32.528380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2d593-1d42-441d-8e62-1913f25f652a admin admin] PATH_INFO: `/v3/users/cc6095af50fa473cbdede2213b0000b2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:32.529722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2d593-1d42-441d-8e62-1913f25f652a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc6095af50fa473cbdede2213b0000b2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:32.531835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2d593-1d42-441d-8e62-1913f25f652a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:32.847566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b1d2d593-1d42-441d-8e62-1913f25f652a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc6095af50fa473cbdede2213b0000b2'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:32.848686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b1d2d593-1d42-441d-8e62-1913f25f652a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc6095af50fa473cbdede2213b0000b2'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:32.850933 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b1d2d593-1d42-441d-8e62-1913f25f652a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc6095af50fa473cbdede2213b0000b2'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:32.862974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b1d2d593-1d42-441d-8e62-1913f25f652a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc6095af50fa473cbdede2213b0000b2'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:32.882372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b1d2d593-1d42-441d-8e62-1913f25f652a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc6095af50fa473cbdede2213b0000b2'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:33.030537 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1114/2240] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:07:32 2022] DELETE /identity/v3/users/cc6095af50fa473cbdede2213b0000b2 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:07:33.035203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb308999-a008-4f40-a0ab-44bf3488123d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:33.035764 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb308999-a008-4f40-a0ab-44bf3488123d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:33.035860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb308999-a008-4f40-a0ab-44bf3488123d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:33.049295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:33.056027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-fb308999-a008-4f40-a0ab-44bf3488123d None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:33.057288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-fb308999-a008-4f40-a0ab-44bf3488123d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:33.118536 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1127/2241] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:07:33.130409 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2d593-1d42-441d-8e62-1913f25f652a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:33.131901 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b1d2d593-1d42-441d-8e62-1913f25f652a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:33.164894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:33.192112 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2d593-1d42-441d-8e62-1913f25f652a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:33.197225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd62e2-78ee-49c6-969c-04726e272756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:33.198002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd62e2-78ee-49c6-969c-04726e272756 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:33.198410 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd62e2-78ee-49c6-969c-04726e272756 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:33.198783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd62e2-78ee-49c6-969c-04726e272756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:33.210509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd62e2-78ee-49c6-969c-04726e272756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:33.213123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd62e2-78ee-49c6-969c-04726e272756 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:33.230051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1115/2242] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:33.713140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb308999-a008-4f40-a0ab-44bf3488123d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:33.715101 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-fb308999-a008-4f40-a0ab-44bf3488123d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:33.731038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb308999-a008-4f40-a0ab-44bf3488123d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:33.735642 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d2001-6a80-4712-9b3e-4326afe680d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:33.736622 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d2001-6a80-4712-9b3e-4326afe680d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:33.737082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d2001-6a80-4712-9b3e-4326afe680d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:33.737528 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d2001-6a80-4712-9b3e-4326afe680d3 admin admin] PATH_INFO: `/v3/users/a124a80fc9d14f5bbcf63370887b08e1` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:33.739247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d2001-6a80-4712-9b3e-4326afe680d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a124a80fc9d14f5bbcf63370887b08e1)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:33.742338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d2001-6a80-4712-9b3e-4326afe680d3 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:33.810850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd62e2-78ee-49c6-969c-04726e272756 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:33.812703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-68bd62e2-78ee-49c6-969c-04726e272756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:33.844218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd62e2-78ee-49c6-969c-04726e272756 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:33.849734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e797d90-6b24-47fb-89ae-d2eff940540b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:33.850444 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e797d90-6b24-47fb-89ae-d2eff940540b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:33.850739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e797d90-6b24-47fb-89ae-d2eff940540b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:33.851048 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e797d90-6b24-47fb-89ae-d2eff940540b admin admin] PATH_INFO: `/v3/projects/5cc2bf83118e4b22a3fc0678236d2018` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:33.852436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e797d90-6b24-47fb-89ae-d2eff940540b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc2bf83118e4b22a3fc0678236d2018)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:33.854361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e797d90-6b24-47fb-89ae-d2eff940540b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:33.954039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-360d2001-6a80-4712-9b3e-4326afe680d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a124a80fc9d14f5bbcf63370887b08e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:33.963380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-360d2001-6a80-4712-9b3e-4326afe680d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a124a80fc9d14f5bbcf63370887b08e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:33.977220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3e797d90-6b24-47fb-89ae-d2eff940540b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc2bf83118e4b22a3fc0678236d2018', 'request_id': 'req-3e797d90-6b24-47fb-89ae-d2eff940540b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:33.983921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3e797d90-6b24-47fb-89ae-d2eff940540b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc2bf83118e4b22a3fc0678236d2018', 'request_id': 'req-3e797d90-6b24-47fb-89ae-d2eff940540b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:33.990510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3e797d90-6b24-47fb-89ae-d2eff940540b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc2bf83118e4b22a3fc0678236d2018', 'request_id': 'req-3e797d90-6b24-47fb-89ae-d2eff940540b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:34.074968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-360d2001-6a80-4712-9b3e-4326afe680d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a124a80fc9d14f5bbcf63370887b08e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:34.088762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-360d2001-6a80-4712-9b3e-4326afe680d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a124a80fc9d14f5bbcf63370887b08e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:34.089598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-360d2001-6a80-4712-9b3e-4326afe680d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a124a80fc9d14f5bbcf63370887b08e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:34.097257 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1128/2243] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:07:33 2022] DELETE /identity/v3/users/a124a80fc9d14f5bbcf63370887b08e1 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:07:34.102417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f592b-0f65-4cfc-99cd-80469a49ebf6 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:34.103004 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f592b-0f65-4cfc-99cd-80469a49ebf6 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:34.103388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f592b-0f65-4cfc-99cd-80469a49ebf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:34.124266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-829f592b-0f65-4cfc-99cd-80469a49ebf6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:34.125568 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-829f592b-0f65-4cfc-99cd-80469a49ebf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:34.192508 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1129/2244] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:07:34.203236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f592b-0f65-4cfc-99cd-80469a49ebf6 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:34.204855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-829f592b-0f65-4cfc-99cd-80469a49ebf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:34.228868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:34.231264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3e797d90-6b24-47fb-89ae-d2eff940540b admin admin] The token cache is being invalidate because project 5cc2bf83118e4b22a3fc0678236d2018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:07:34.231600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3e797d90-6b24-47fb-89ae-d2eff940540b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:34.232288 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3e797d90-6b24-47fb-89ae-d2eff940540b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:34.233855 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1116/2245] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:07:33 2022] DELETE /identity/v3/projects/5cc2bf83118e4b22a3fc0678236d2018 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:34.241745 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ba7d3-17e9-4545-b584-705109f7c983 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:34.241956 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ba7d3-17e9-4545-b584-705109f7c983 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:34.242242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ba7d3-17e9-4545-b584-705109f7c983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:34.278420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-3a2ba7d3-17e9-4545-b584-705109f7c983 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:34.280733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3a2ba7d3-17e9-4545-b584-705109f7c983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:34.286179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f592b-0f65-4cfc-99cd-80469a49ebf6 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:34.292085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942f0c5-b23e-4f84-b091-4809869c514b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:34.292888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942f0c5-b23e-4f84-b091-4809869c514b service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:34.293144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942f0c5-b23e-4f84-b091-4809869c514b service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:34.293454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942f0c5-b23e-4f84-b091-4809869c514b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:34.295519 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c942f0c5-b23e-4f84-b091-4809869c514b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:34.297298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1117/2246] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:34.304934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73abf23-9c40-4baf-81bb-e3ad55dd2647 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:34.305469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73abf23-9c40-4baf-81bb-e3ad55dd2647 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:34.305925 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73abf23-9c40-4baf-81bb-e3ad55dd2647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:34.317405 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942f0c5-b23e-4f84-b091-4809869c514b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:34.319128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942f0c5-b23e-4f84-b091-4809869c514b service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:34.330098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1130/2247] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:07:34.331021 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942f0c5-b23e-4f84-b091-4809869c514b service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:34.332433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c942f0c5-b23e-4f84-b091-4809869c514b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:34.335538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-b73abf23-9c40-4baf-81bb-e3ad55dd2647 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:34.336911 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b73abf23-9c40-4baf-81bb-e3ad55dd2647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:34.345747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942f0c5-b23e-4f84-b091-4809869c514b service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:34.352225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1118/2248] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:34.355419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73abf23-9c40-4baf-81bb-e3ad55dd2647 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:34.357965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f59733-52db-4fd0-9dc6-e989809c08c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:34.358843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f59733-52db-4fd0-9dc6-e989809c08c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:34.359307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f59733-52db-4fd0-9dc6-e989809c08c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:34.359768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f59733-52db-4fd0-9dc6-e989809c08c2 admin admin] PATH_INFO: `/v3/users/02da7ee5801e424aac67a6b40ef46932` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:34.361239 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f59733-52db-4fd0-9dc6-e989809c08c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02da7ee5801e424aac67a6b40ef46932)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:34.363647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f59733-52db-4fd0-9dc6-e989809c08c2 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:34.374992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73abf23-9c40-4baf-81bb-e3ad55dd2647 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:34.382431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b869b39-ddc8-4633-86ab-0921e48b2f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:34.383347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b869b39-ddc8-4633-86ab-0921e48b2f00 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:34.383928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b869b39-ddc8-4633-86ab-0921e48b2f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:34.384260 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b869b39-ddc8-4633-86ab-0921e48b2f00 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:34.385148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b869b39-ddc8-4633-86ab-0921e48b2f00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:07:34.385575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b869b39-ddc8-4633-86ab-0921e48b2f00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:34.388140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b869b39-ddc8-4633-86ab-0921e48b2f00 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:34.406579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1119/2249] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:07:34.410766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b869b39-ddc8-4633-86ab-0921e48b2f00 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:34.423833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b869b39-ddc8-4633-86ab-0921e48b2f00 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:34.429307 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafd2a0-8bd9-45d2-a5b4-144e0b8e78da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:34.430030 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafd2a0-8bd9-45d2-a5b4-144e0b8e78da admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:34.432419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafd2a0-8bd9-45d2-a5b4-144e0b8e78da admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:34.432419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafd2a0-8bd9-45d2-a5b4-144e0b8e78da admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:34.432419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafd2a0-8bd9-45d2-a5b4-144e0b8e78da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:34.433298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafd2a0-8bd9-45d2-a5b4-144e0b8e78da admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:34.581766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1120/2250] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:07:34 2022] POST /identity/v3/projects => generated 371 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:07:34.589393 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafd2a0-8bd9-45d2-a5b4-144e0b8e78da admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:34.668512 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafd2a0-8bd9-45d2-a5b4-144e0b8e78da admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:34.674394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a8d97-73a5-4711-9736-fbf41451f95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:34.675024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a8d97-73a5-4711-9736-fbf41451f95b admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:34.675445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a8d97-73a5-4711-9736-fbf41451f95b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:34.675738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a8d97-73a5-4711-9736-fbf41451f95b admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:34.676619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a8d97-73a5-4711-9736-fbf41451f95b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:34.678698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a8d97-73a5-4711-9736-fbf41451f95b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:34.691768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:34.723249 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d2f59733-52db-4fd0-9dc6-e989809c08c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02da7ee5801e424aac67a6b40ef46932'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:34.743518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d2f59733-52db-4fd0-9dc6-e989809c08c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02da7ee5801e424aac67a6b40ef46932'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:34.798874 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1121/2251] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:07:34 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:34.803337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a8d97-73a5-4711-9736-fbf41451f95b admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:34.816703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a8d97-73a5-4711-9736-fbf41451f95b admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:34.820935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d2f59733-52db-4fd0-9dc6-e989809c08c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02da7ee5801e424aac67a6b40ef46932'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:34.826920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d2e2b-40c3-43b1-baf6-a7ad8cdf9b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:34.827245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d2e2b-40c3-43b1-baf6-a7ad8cdf9b95 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:34.827683 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d2e2b-40c3-43b1-baf6-a7ad8cdf9b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:34.827817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d2e2b-40c3-43b1-baf6-a7ad8cdf9b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:34.828635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d2e2b-40c3-43b1-baf6-a7ad8cdf9b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:34.841369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d2e2b-40c3-43b1-baf6-a7ad8cdf9b95 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:34.848814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d2f59733-52db-4fd0-9dc6-e989809c08c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02da7ee5801e424aac67a6b40ef46932'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:34.851203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d2f59733-52db-4fd0-9dc6-e989809c08c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02da7ee5801e424aac67a6b40ef46932'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:34.861889 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1122/2252] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:07:34.862737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d2e2b-40c3-43b1-baf6-a7ad8cdf9b95 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:34.866624 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1131/2253] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:07:34 2022] DELETE /identity/v3/users/02da7ee5801e424aac67a6b40ef46932 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:34.867182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-016d2e2b-40c3-43b1-baf6-a7ad8cdf9b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:34.883132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7faf67-a0bb-4d14-871c-fe355759dde5 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:34.883456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7faf67-a0bb-4d14-871c-fe355759dde5 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:34.883848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7faf67-a0bb-4d14-871c-fe355759dde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:34.918039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d2e2b-40c3-43b1-baf6-a7ad8cdf9b95 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:34.958549 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-1d7faf67-a0bb-4d14-871c-fe355759dde5 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:34.960300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1d7faf67-a0bb-4d14-871c-fe355759dde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:35.062885 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bd24d-31b5-42df-87f6-3b41b7d15ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:35.064301 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bd24d-31b5-42df-87f6-3b41b7d15ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:35.064666 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bd24d-31b5-42df-87f6-3b41b7d15ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:35.065052 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bd24d-31b5-42df-87f6-3b41b7d15ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:35.066077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bd24d-31b5-42df-87f6-3b41b7d15ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:35.070988 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1132/2254] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:35.071177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bd24d-31b5-42df-87f6-3b41b7d15ed8 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:35.071274 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7faf67-a0bb-4d14-871c-fe355759dde5 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:35.071274 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1d7faf67-a0bb-4d14-871c-fe355759dde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:35.084831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7faf67-a0bb-4d14-871c-fe355759dde5 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:35.094335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:35.094597 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1123/2255] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:35.096273 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bd24d-31b5-42df-87f6-3b41b7d15ed8 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:35.098125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8b1bd24d-31b5-42df-87f6-3b41b7d15ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:35.098623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:35.099547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:35.099993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] PATH_INFO: `/v3/projects/30cfba60d2fe4fdb8d208be6e8726475` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:35.101642 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=30cfba60d2fe4fdb8d208be6e8726475)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:35.103607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:35.166940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bd24d-31b5-42df-87f6-3b41b7d15ed8 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:35.171427 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6aa2c-158c-4e43-ba24-3dfba63d08e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:35.172085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6aa2c-158c-4e43-ba24-3dfba63d08e5 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:35.172372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6aa2c-158c-4e43-ba24-3dfba63d08e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:35.172684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6aa2c-158c-4e43-ba24-3dfba63d08e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:35.187745 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6aa2c-158c-4e43-ba24-3dfba63d08e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:35.189225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6aa2c-158c-4e43-ba24-3dfba63d08e5 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:35.200406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1124/2256] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:35.201494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6aa2c-158c-4e43-ba24-3dfba63d08e5 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:35.213061 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6aa2c-158c-4e43-ba24-3dfba63d08e5 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:35.223626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be0af8-3bfe-440e-9451-03c53131bb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:35.229362 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be0af8-3bfe-440e-9451-03c53131bb16 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:35.229362 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be0af8-3bfe-440e-9451-03c53131bb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:35.229362 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be0af8-3bfe-440e-9451-03c53131bb16 admin admin] PATH_INFO: `/v3/projects/a9f0ab2fc6e64eada5f55a0f0a835073/users/2fc5fd88556145fda1fb0067d5e9adcc/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:35.243260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30cfba60d2fe4fdb8d208be6e8726475', 'request_id': 'req-090ca257-af23-418f-b2f8-872e709dd0ce'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:35.248503 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be0af8-3bfe-440e-9451-03c53131bb16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9f0ab2fc6e64eada5f55a0f0a835073, user_id=2fc5fd88556145fda1fb0067d5e9adcc, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:35.252071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30cfba60d2fe4fdb8d208be6e8726475', 'request_id': 'req-090ca257-af23-418f-b2f8-872e709dd0ce'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:35.252783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be0af8-3bfe-440e-9451-03c53131bb16 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:35.385036 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30cfba60d2fe4fdb8d208be6e8726475', 'request_id': 'req-090ca257-af23-418f-b2f8-872e709dd0ce'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:35.395604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1125/2257] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:35 2022] PUT /identity/v3/projects/a9f0ab2fc6e64eada5f55a0f0a835073/users/2fc5fd88556145fda1fb0067d5e9adcc/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:35.402602 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f2a69-8bf7-475b-ba30-cf7ef34c6983 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:35.402913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] The token cache is being invalidate because project 30cfba60d2fe4fdb8d208be6e8726475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:07:35.403159 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f2a69-8bf7-475b-ba30-cf7ef34c6983 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:35.403317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:35.403630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f2a69-8bf7-475b-ba30-cf7ef34c6983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:35.406028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:35.409001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1133/2258] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:07:35 2022] DELETE /identity/v3/projects/30cfba60d2fe4fdb8d208be6e8726475 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:35.417565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:35.419029 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:35.430781 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ba2f2a69-8bf7-475b-ba30-cf7ef34c6983 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:35.432475 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ba2f2a69-8bf7-475b-ba30-cf7ef34c6983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:35.493550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ca257-af23-418f-b2f8-872e709dd0ce admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:35.507972 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1126/2259] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:35.516629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f2a69-8bf7-475b-ba30-cf7ef34c6983 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:35.516834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:35.521932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ba2f2a69-8bf7-475b-ba30-cf7ef34c6983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:35.581762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40a867-9f85-4a25-81ea-407fffb43419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:35.582662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40a867-9f85-4a25-81ea-407fffb43419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:35.583046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40a867-9f85-4a25-81ea-407fffb43419 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:35.583941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40a867-9f85-4a25-81ea-407fffb43419 admin admin] PATH_INFO: `/v3/users/8a96896672244b4c91492535286c42bc` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:35.585465 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40a867-9f85-4a25-81ea-407fffb43419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a96896672244b4c91492535286c42bc)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:35.587954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40a867-9f85-4a25-81ea-407fffb43419 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:35.619130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f2a69-8bf7-475b-ba30-cf7ef34c6983 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:35.630671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b60221-4f08-4306-b74e-942621b142b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:35.631456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b60221-4f08-4306-b74e-942621b142b0 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:35.631780 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b60221-4f08-4306-b74e-942621b142b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:35.632254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b60221-4f08-4306-b74e-942621b142b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:35.649358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b60221-4f08-4306-b74e-942621b142b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:35.651038 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b60221-4f08-4306-b74e-942621b142b0 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:35.666107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1127/2260] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:07:35.806097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b60221-4f08-4306-b74e-942621b142b0 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:35.808497 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-61b60221-4f08-4306-b74e-942621b142b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:35.820406 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-4b40a867-9f85-4a25-81ea-407fffb43419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a96896672244b4c91492535286c42bc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:35.828532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b60221-4f08-4306-b74e-942621b142b0 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:35.841216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-4b40a867-9f85-4a25-81ea-407fffb43419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a96896672244b4c91492535286c42bc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:35.862547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:35.863269 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:35.864196 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:35.865555 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9 admin admin] PATH_INFO: `/v3/projects/ad6d7ffaa7104eb68ce544d95dd4605e` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:35.867645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad6d7ffaa7104eb68ce544d95dd4605e)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:35.871986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:36.017706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-4b40a867-9f85-4a25-81ea-407fffb43419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a96896672244b4c91492535286c42bc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:36.035166 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-4b40a867-9f85-4a25-81ea-407fffb43419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a96896672244b4c91492535286c42bc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:36.035974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-4b40a867-9f85-4a25-81ea-407fffb43419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a96896672244b4c91492535286c42bc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:36.047627 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1134/2261] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:07:35 2022] DELETE /identity/v3/users/8a96896672244b4c91492535286c42bc => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:36.060177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:36.125432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad6d7ffaa7104eb68ce544d95dd4605e', 'request_id': 'req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:36.153142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad6d7ffaa7104eb68ce544d95dd4605e', 'request_id': 'req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:36.170873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad6d7ffaa7104eb68ce544d95dd4605e', 'request_id': 'req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:36.341114 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9 admin admin] The token cache is being invalidate because project ad6d7ffaa7104eb68ce544d95dd4605e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:07:36.341114 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:36.342114 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-544fd1ce-0064-43dd-9154-8f0ec8c1f4e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:36.344014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1128/2262] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:07:35 2022] DELETE /identity/v3/projects/ad6d7ffaa7104eb68ce544d95dd4605e => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:07:36.349866 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd9cb9-6a12-4e45-ab83-b89ed020a996 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:36.350467 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd9cb9-6a12-4e45-ab83-b89ed020a996 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:36.350571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd9cb9-6a12-4e45-ab83-b89ed020a996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:36.381765 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-eccd9cb9-6a12-4e45-ab83-b89ed020a996 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:36.383098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-eccd9cb9-6a12-4e45-ab83-b89ed020a996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:36.465086 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1135/2263] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:07:36.473270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29506c0-c8ca-4b35-bc1a-bbfb4b6cebd6 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:36.474646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29506c0-c8ca-4b35-bc1a-bbfb4b6cebd6 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:36.474646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29506c0-c8ca-4b35-bc1a-bbfb4b6cebd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:36.531873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-c29506c0-c8ca-4b35-bc1a-bbfb4b6cebd6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:36.535961 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c29506c0-c8ca-4b35-bc1a-bbfb4b6cebd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:36.557203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1129/2264] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:07:36.558866 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd9cb9-6a12-4e45-ab83-b89ed020a996 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:36.576286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd9cb9-6a12-4e45-ab83-b89ed020a996 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:36.583744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5207334-84c3-4114-b1d0-adcf52a7f761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:36.585030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5207334-84c3-4114-b1d0-adcf52a7f761 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:36.587591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5207334-84c3-4114-b1d0-adcf52a7f761 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:36.587591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5207334-84c3-4114-b1d0-adcf52a7f761 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:36.587591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5207334-84c3-4114-b1d0-adcf52a7f761 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:07:36.587591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5207334-84c3-4114-b1d0-adcf52a7f761 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:36.589294 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5207334-84c3-4114-b1d0-adcf52a7f761 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:36.607522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1136/2265] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:07:36.612551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29506c0-c8ca-4b35-bc1a-bbfb4b6cebd6 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:36.627701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29506c0-c8ca-4b35-bc1a-bbfb4b6cebd6 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:36.639679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e9aab-713b-4719-bb4b-5556b4615fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:36.644908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e9aab-713b-4719-bb4b-5556b4615fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:36.644908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e9aab-713b-4719-bb4b-5556b4615fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:36.644908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e9aab-713b-4719-bb4b-5556b4615fa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:36.644908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e9aab-713b-4719-bb4b-5556b4615fa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:36.646896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e9aab-713b-4719-bb4b-5556b4615fa3 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:36.746788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:36.788567 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5207334-84c3-4114-b1d0-adcf52a7f761 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:36.790298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1130/2266] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:07:36 2022] POST /identity/v3/projects => generated 369 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:36.794302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b5207334-84c3-4114-b1d0-adcf52a7f761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:36.794789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e9aab-713b-4719-bb4b-5556b4615fa3 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:36.821949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5207334-84c3-4114-b1d0-adcf52a7f761 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:36.892323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c24480-a39f-4fe1-9310-07fd672d7be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:36.893010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c24480-a39f-4fe1-9310-07fd672d7be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:36.893362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c24480-a39f-4fe1-9310-07fd672d7be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:36.893712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c24480-a39f-4fe1-9310-07fd672d7be6 admin admin] PATH_INFO: `/v3/projects/8187a4395efb4338ae1fd75cbb5ce775` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:36.895348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c24480-a39f-4fe1-9310-07fd672d7be6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8187a4395efb4338ae1fd75cbb5ce775)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:36.898724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c24480-a39f-4fe1-9310-07fd672d7be6 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:36.913623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:36.930894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e9aab-713b-4719-bb4b-5556b4615fa3 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:36.937469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e768d55-0d59-4b58-a991-12a1869a12cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:36.938583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e768d55-0d59-4b58-a991-12a1869a12cb admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:36.939036 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e768d55-0d59-4b58-a991-12a1869a12cb admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:36.939442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e768d55-0d59-4b58-a991-12a1869a12cb admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:36.940359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e768d55-0d59-4b58-a991-12a1869a12cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:36.943554 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e768d55-0d59-4b58-a991-12a1869a12cb admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:37.055535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1131/2267] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:07:36 2022] POST /identity/v3/users => generated 285 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:07:37.060595 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e768d55-0d59-4b58-a991-12a1869a12cb admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:37.086064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e768d55-0d59-4b58-a991-12a1869a12cb admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:37.101935 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7cc1e-c806-4d57-8979-d43d3c4cbb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:37.102898 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7cc1e-c806-4d57-8979-d43d3c4cbb8a admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:37.103187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7cc1e-c806-4d57-8979-d43d3c4cbb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:37.103601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7cc1e-c806-4d57-8979-d43d3c4cbb8a admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:37.104331 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7cc1e-c806-4d57-8979-d43d3c4cbb8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:37.109866 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7cc1e-c806-4d57-8979-d43d3c4cbb8a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:37.139162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1132/2268] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:37.156719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7cc1e-c806-4d57-8979-d43d3c4cbb8a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:37.156831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-69c24480-a39f-4fe1-9310-07fd672d7be6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8187a4395efb4338ae1fd75cbb5ce775', 'request_id': 'req-69c24480-a39f-4fe1-9310-07fd672d7be6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:37.163045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7cc1e-c806-4d57-8979-d43d3c4cbb8a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:37.169687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-69c24480-a39f-4fe1-9310-07fd672d7be6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8187a4395efb4338ae1fd75cbb5ce775', 'request_id': 'req-69c24480-a39f-4fe1-9310-07fd672d7be6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:37.293778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdafea4-821d-412f-99d0-930da076196d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:37.294555 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdafea4-821d-412f-99d0-930da076196d admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:37.298198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdafea4-821d-412f-99d0-930da076196d admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:37.298198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdafea4-821d-412f-99d0-930da076196d admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:37.298198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdafea4-821d-412f-99d0-930da076196d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:37.300829 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdafea4-821d-412f-99d0-930da076196d admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:37.319089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-69c24480-a39f-4fe1-9310-07fd672d7be6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8187a4395efb4338ae1fd75cbb5ce775', 'request_id': 'req-69c24480-a39f-4fe1-9310-07fd672d7be6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:37.319296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1133/2269] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:37.320488 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdafea4-821d-412f-99d0-930da076196d admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:37.334892 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdafea4-821d-412f-99d0-930da076196d admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:37.342138 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc34a23-2065-4876-adca-a897a1af51b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:37.343566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc34a23-2065-4876-adca-a897a1af51b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:37.344052 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc34a23-2065-4876-adca-a897a1af51b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:37.344568 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc34a23-2065-4876-adca-a897a1af51b4 admin admin] PATH_INFO: `/v3/projects/e9eba5436c0c4f52b5c4e7412176a751/users/c6984944b2ee4f6bb958a2f4a9f08172/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:37.363485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-69c24480-a39f-4fe1-9310-07fd672d7be6 admin admin] The token cache is being invalidate because project 8187a4395efb4338ae1fd75cbb5ce775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:07:37.366772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-69c24480-a39f-4fe1-9310-07fd672d7be6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:37.367667 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc34a23-2065-4876-adca-a897a1af51b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9eba5436c0c4f52b5c4e7412176a751, user_id=c6984944b2ee4f6bb958a2f4a9f08172, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:37.368190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-69c24480-a39f-4fe1-9310-07fd672d7be6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:37.370945 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc34a23-2065-4876-adca-a897a1af51b4 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:37.371787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1137/2270] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:07:36 2022] DELETE /identity/v3/projects/8187a4395efb4338ae1fd75cbb5ce775 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:37.376716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffe198-3159-4b8e-a40c-2683ac235a0b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:37.377224 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffe198-3159-4b8e-a40c-2683ac235a0b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:37.377646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffe198-3159-4b8e-a40c-2683ac235a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:37.399512 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-daffe198-3159-4b8e-a40c-2683ac235a0b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:37.400759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-daffe198-3159-4b8e-a40c-2683ac235a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:37.409221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1138/2271] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:37.421104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6058a6c-5d5c-446c-9783-620195640979 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:37.421595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6058a6c-5d5c-446c-9783-620195640979 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:37.421873 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6058a6c-5d5c-446c-9783-620195640979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:37.441669 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-f6058a6c-5d5c-446c-9783-620195640979 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:37.443410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f6058a6c-5d5c-446c-9783-620195640979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:37.451414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1139/2272] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:37.455903 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6058a6c-5d5c-446c-9783-620195640979 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:37.467074 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6058a6c-5d5c-446c-9783-620195640979 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:37.472548 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741a2ea-f3a7-4e36-8b70-7a659488c02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:37.473546 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741a2ea-f3a7-4e36-8b70-7a659488c02a admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:37.474180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741a2ea-f3a7-4e36-8b70-7a659488c02a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:37.474606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741a2ea-f3a7-4e36-8b70-7a659488c02a admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:37.475484 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741a2ea-f3a7-4e36-8b70-7a659488c02a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:07:37.475944 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741a2ea-f3a7-4e36-8b70-7a659488c02a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:37.477756 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741a2ea-f3a7-4e36-8b70-7a659488c02a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:37.497623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1134/2273] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:37 2022] PUT /identity/v3/projects/e9eba5436c0c4f52b5c4e7412176a751/users/c6984944b2ee4f6bb958a2f4a9f08172/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:37.500830 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1140/2274] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:07:37.506014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc34a23-2065-4876-adca-a897a1af51b4 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:37.509043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f71038-6689-4e94-a10c-73b179a33264 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:37.512591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f71038-6689-4e94-a10c-73b179a33264 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:37.513093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f71038-6689-4e94-a10c-73b179a33264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:37.554013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-f6f71038-6689-4e94-a10c-73b179a33264 None None] MFA Rules not processed for user `c6984944b2ee4f6bb958a2f4a9f08172`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:37.556358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f6f71038-6689-4e94-a10c-73b179a33264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:37.624879 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc34a23-2065-4876-adca-a897a1af51b4 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:37.631432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac1f21-2fac-42c5-9e4d-0598dbdb5cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:37.632289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac1f21-2fac-42c5-9e4d-0598dbdb5cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:37.632580 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac1f21-2fac-42c5-9e4d-0598dbdb5cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:37.632922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac1f21-2fac-42c5-9e4d-0598dbdb5cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:37.633617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac1f21-2fac-42c5-9e4d-0598dbdb5cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:37.635575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac1f21-2fac-42c5-9e4d-0598dbdb5cf7 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:37.650470 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1141/2275] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:37.668322 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f71038-6689-4e94-a10c-73b179a33264 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:37.669728 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f6f71038-6689-4e94-a10c-73b179a33264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:37.752477 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f71038-6689-4e94-a10c-73b179a33264 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:37.758532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56addb-54ec-42b8-bcde-1fec5eae8190 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b153935910c142aa80792e111872318a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:37.759275 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56addb-54ec-42b8-bcde-1fec5eae8190 service glance] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:37.759590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56addb-54ec-42b8-bcde-1fec5eae8190 service glance] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:37.759860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56addb-54ec-42b8-bcde-1fec5eae8190 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:37.777226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56addb-54ec-42b8-bcde-1fec5eae8190 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:37.779343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56addb-54ec-42b8-bcde-1fec5eae8190 service glance] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:37.785034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1135/2276] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:07:37 2022] POST /identity/v3/projects => generated 381 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:37.790096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac1f21-2fac-42c5-9e4d-0598dbdb5cf7 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:37.791408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1142/2277] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:07:37.871360 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac1f21-2fac-42c5-9e4d-0598dbdb5cf7 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:37.876106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ce36c-4f6c-4a14-8452-18b768064af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:37.878081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ce36c-4f6c-4a14-8452-18b768064af3 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:37.878081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ce36c-4f6c-4a14-8452-18b768064af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:37.878081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ce36c-4f6c-4a14-8452-18b768064af3 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:37.878081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ce36c-4f6c-4a14-8452-18b768064af3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:37.880295 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ce36c-4f6c-4a14-8452-18b768064af3 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:37.937629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:38.007515 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1136/2278] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:07:37 2022] POST /identity/v3/users => generated 291 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:07:38.010109 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56addb-54ec-42b8-bcde-1fec5eae8190 service glance] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:38.026482 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56addb-54ec-42b8-bcde-1fec5eae8190 service glance] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:38.031595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044edc3f-4441-424a-bbbb-bb203acc91d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:38.035645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044edc3f-4441-424a-bbbb-bb203acc91d8 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:38.036127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044edc3f-4441-424a-bbbb-bb203acc91d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:38.037055 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044edc3f-4441-424a-bbbb-bb203acc91d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:38.038593 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044edc3f-4441-424a-bbbb-bb203acc91d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:38.040453 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044edc3f-4441-424a-bbbb-bb203acc91d8 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:38.061971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1143/2279] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:38.066183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ce36c-4f6c-4a14-8452-18b768064af3 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:38.085223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ce36c-4f6c-4a14-8452-18b768064af3 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:38.101449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34537f-f6b5-496e-975e-743180b64c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:38.102871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34537f-f6b5-496e-975e-743180b64c3a admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:38.103350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34537f-f6b5-496e-975e-743180b64c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:38.103935 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34537f-f6b5-496e-975e-743180b64c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:38.105006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c34537f-f6b5-496e-975e-743180b64c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:38.106929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c34537f-f6b5-496e-975e-743180b64c3a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:38.123784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1137/2280] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:38.127664 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044edc3f-4441-424a-bbbb-bb203acc91d8 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:38.140001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044edc3f-4441-424a-bbbb-bb203acc91d8 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:38.154669 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a116ede-141a-4186-8bab-6aa6874b10c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:38.155377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a116ede-141a-4186-8bab-6aa6874b10c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:38.155688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a116ede-141a-4186-8bab-6aa6874b10c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:38.155961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a116ede-141a-4186-8bab-6aa6874b10c3 admin admin] PATH_INFO: `/v3/projects/f53ee2df7667452a87e2b377ad1b1924/users/7d98658dda214820b8cf5d737f16dbba/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:38.179762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a116ede-141a-4186-8bab-6aa6874b10c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f53ee2df7667452a87e2b377ad1b1924, user_id=7d98658dda214820b8cf5d737f16dbba, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:38.181332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a116ede-141a-4186-8bab-6aa6874b10c3 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:38.277590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1144/2281] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:38 2022] PUT /identity/v3/projects/f53ee2df7667452a87e2b377ad1b1924/users/7d98658dda214820b8cf5d737f16dbba/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:38.289164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0062f-d15f-40d9-ba03-e84ae13895b5 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:38.289498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0062f-d15f-40d9-ba03-e84ae13895b5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:38.289749 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0062f-d15f-40d9-ba03-e84ae13895b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:38.327931 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:38.332135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-20a0062f-d15f-40d9-ba03-e84ae13895b5 None None] MFA Rules not processed for user `7d98658dda214820b8cf5d737f16dbba`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:38.333444 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-20a0062f-d15f-40d9-ba03-e84ae13895b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:38.471703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1138/2282] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:38.476413 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a116ede-141a-4186-8bab-6aa6874b10c3 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:38.579624 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a116ede-141a-4186-8bab-6aa6874b10c3 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:38.585140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef761300-a79c-4f34-8a62-3dd659cd25d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:38.585780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef761300-a79c-4f34-8a62-3dd659cd25d4 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:38.586057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef761300-a79c-4f34-8a62-3dd659cd25d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:38.586332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef761300-a79c-4f34-8a62-3dd659cd25d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:38.587017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef761300-a79c-4f34-8a62-3dd659cd25d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:38.589125 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef761300-a79c-4f34-8a62-3dd659cd25d4 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:38.738177 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1145/2283] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:07:38 2022] POST /identity/v3/projects => generated 381 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:38.742989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0062f-d15f-40d9-ba03-e84ae13895b5 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:38.762934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0062f-d15f-40d9-ba03-e84ae13895b5 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:38.922533 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a46419-e94e-4f70-a212-1af7fa4f339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:38.923309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a46419-e94e-4f70-a212-1af7fa4f339e admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:38.923779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a46419-e94e-4f70-a212-1af7fa4f339e admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:38.924121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a46419-e94e-4f70-a212-1af7fa4f339e admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:38.925071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a46419-e94e-4f70-a212-1af7fa4f339e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:38.928099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a46419-e94e-4f70-a212-1af7fa4f339e admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:39.098288 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1139/2284] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:07:38 2022] POST /identity/v3/users => generated 297 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:39.099077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef761300-a79c-4f34-8a62-3dd659cd25d4 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:39.108259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef761300-a79c-4f34-8a62-3dd659cd25d4 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:39.123302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561aa3c-f255-4131-9a1a-f8563fd91211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:39.123584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561aa3c-f255-4131-9a1a-f8563fd91211 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:39.123799 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561aa3c-f255-4131-9a1a-f8563fd91211 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:39.124075 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561aa3c-f255-4131-9a1a-f8563fd91211 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:39.125619 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561aa3c-f255-4131-9a1a-f8563fd91211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:39.131746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561aa3c-f255-4131-9a1a-f8563fd91211 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:39.181188 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1146/2285] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:39.185222 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a46419-e94e-4f70-a212-1af7fa4f339e admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:39.201541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a46419-e94e-4f70-a212-1af7fa4f339e admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:39.208982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f4efb-7acb-41b1-a0b1-e2fae16fdfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:39.210005 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f4efb-7acb-41b1-a0b1-e2fae16fdfdb admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:39.210332 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f4efb-7acb-41b1-a0b1-e2fae16fdfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:39.210579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f4efb-7acb-41b1-a0b1-e2fae16fdfdb admin admin] PATH_INFO: `/v3/projects/cd88f24af1974b9c9a961e202c1b28bc/users/d37ae214fccb43d295d7a94ca76c30cf/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:39.241722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f4efb-7acb-41b1-a0b1-e2fae16fdfdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd88f24af1974b9c9a961e202c1b28bc, user_id=d37ae214fccb43d295d7a94ca76c30cf, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:39.245835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f4efb-7acb-41b1-a0b1-e2fae16fdfdb admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:39.385824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1140/2286] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:39 2022] PUT /identity/v3/projects/cd88f24af1974b9c9a961e202c1b28bc/users/d37ae214fccb43d295d7a94ca76c30cf/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:39.389816 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561aa3c-f255-4131-9a1a-f8563fd91211 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:39.411029 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561aa3c-f255-4131-9a1a-f8563fd91211 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:39.509929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c07971-76e9-4920-9b1c-f2ec39e9e584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:39.511229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c07971-76e9-4920-9b1c-f2ec39e9e584 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:39.511541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c07971-76e9-4920-9b1c-f2ec39e9e584 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:39.511820 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c07971-76e9-4920-9b1c-f2ec39e9e584 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:39.514766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c07971-76e9-4920-9b1c-f2ec39e9e584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:39.519223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c07971-76e9-4920-9b1c-f2ec39e9e584 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:39.545439 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1147/2287] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:07:39.558877 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f4efb-7acb-41b1-a0b1-e2fae16fdfdb admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:39.566443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f4efb-7acb-41b1-a0b1-e2fae16fdfdb admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:39.566443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8bffc-17d2-4d98-b00c-e09ddd277ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:39.566443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8bffc-17d2-4d98-b00c-e09ddd277ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:39.566733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8bffc-17d2-4d98-b00c-e09ddd277ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:39.566899 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8bffc-17d2-4d98-b00c-e09ddd277ea7 admin admin] PATH_INFO: `/v3/projects/cd88f24af1974b9c9a961e202c1b28bc/users/d37ae214fccb43d295d7a94ca76c30cf/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:39.569325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8bffc-17d2-4d98-b00c-e09ddd277ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd88f24af1974b9c9a961e202c1b28bc, user_id=d37ae214fccb43d295d7a94ca76c30cf, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:39.572600 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8bffc-17d2-4d98-b00c-e09ddd277ea7 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:39.593903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1141/2288] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:39 2022] PUT /identity/v3/projects/cd88f24af1974b9c9a961e202c1b28bc/users/d37ae214fccb43d295d7a94ca76c30cf/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:39.606382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476da87-de24-42d2-9552-160e2a55360c None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:39.606670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476da87-de24-42d2-9552-160e2a55360c None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:39.606943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476da87-de24-42d2-9552-160e2a55360c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:39.644277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-4476da87-de24-42d2-9552-160e2a55360c None None] MFA Rules not processed for user `d37ae214fccb43d295d7a94ca76c30cf`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:39.645646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4476da87-de24-42d2-9552-160e2a55360c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:39.804856 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1148/2289] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:39.814267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8bffc-17d2-4d98-b00c-e09ddd277ea7 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:39.819590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-baa8bffc-17d2-4d98-b00c-e09ddd277ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:39.977767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8bffc-17d2-4d98-b00c-e09ddd277ea7 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:39.985703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260c046-5254-49a0-9c2a-c944597b4b4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:39.986977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260c046-5254-49a0-9c2a-c944597b4b4b service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:39.987938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260c046-5254-49a0-9c2a-c944597b4b4b service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:39.988440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260c046-5254-49a0-9c2a-c944597b4b4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:40.010998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260c046-5254-49a0-9c2a-c944597b4b4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:40.015444 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260c046-5254-49a0-9c2a-c944597b4b4b service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:40.043886 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1142/2290] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:40.190504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:40.767037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b27714-2515-4985-942f-1fb3575dc564 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:40.767558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b27714-2515-4985-942f-1fb3575dc564 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:40.769311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b27714-2515-4985-942f-1fb3575dc564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:40.779210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:40.792819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c7b27714-2515-4985-942f-1fb3575dc564 None None] MFA Rules not processed for user `d2738e8ef94a48bb9a0d6b1177598112`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:40.794226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c7b27714-2515-4985-942f-1fb3575dc564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:40.912368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1149/2291] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:41.037788 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260c046-5254-49a0-9c2a-c944597b4b4b service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:41.136758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260c046-5254-49a0-9c2a-c944597b4b4b service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:41.141957 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be3ffc-3274-44b3-b713-70af318091f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:41.142808 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be3ffc-3274-44b3-b713-70af318091f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:41.143134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be3ffc-3274-44b3-b713-70af318091f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:41.143359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be3ffc-3274-44b3-b713-70af318091f2 admin admin] PATH_INFO: `/v3/users/7d98658dda214820b8cf5d737f16dbba` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:41.144621 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be3ffc-3274-44b3-b713-70af318091f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d98658dda214820b8cf5d737f16dbba)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:41.146578 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be3ffc-3274-44b3-b713-70af318091f2 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:41.419938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-94be3ffc-3274-44b3-b713-70af318091f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d98658dda214820b8cf5d737f16dbba'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:41.421150 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-94be3ffc-3274-44b3-b713-70af318091f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d98658dda214820b8cf5d737f16dbba'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:41.425114 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-94be3ffc-3274-44b3-b713-70af318091f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d98658dda214820b8cf5d737f16dbba'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:41.455675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-94be3ffc-3274-44b3-b713-70af318091f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d98658dda214820b8cf5d737f16dbba'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:41.485534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-94be3ffc-3274-44b3-b713-70af318091f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d98658dda214820b8cf5d737f16dbba'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:41.596978 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1143/2292] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:07:41 2022] DELETE /identity/v3/users/7d98658dda214820b8cf5d737f16dbba => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:42.071843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c176e7-44c3-4647-aba7-90365c26fb09 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:42.071843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c176e7-44c3-4647-aba7-90365c26fb09 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:42.071843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c176e7-44c3-4647-aba7-90365c26fb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:42.071843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b9c176e7-44c3-4647-aba7-90365c26fb09 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:42.071843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b9c176e7-44c3-4647-aba7-90365c26fb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:42.071843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1150/2293] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:42.073167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be3ffc-3274-44b3-b713-70af318091f2 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:42.073167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-94be3ffc-3274-44b3-b713-70af318091f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:42.073167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be3ffc-3274-44b3-b713-70af318091f2 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:42.073994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a4192-a7e4-44cf-944f-4b34ef120919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:42.073994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a4192-a7e4-44cf-944f-4b34ef120919 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:42.073994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a4192-a7e4-44cf-944f-4b34ef120919 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:42.073994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a4192-a7e4-44cf-944f-4b34ef120919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:42.075249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a4192-a7e4-44cf-944f-4b34ef120919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:42.075249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a4192-a7e4-44cf-944f-4b34ef120919 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:42.075249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1144/2294] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:42.144978 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:42.572616 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c176e7-44c3-4647-aba7-90365c26fb09 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:42.574480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b9c176e7-44c3-4647-aba7-90365c26fb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:42.599016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c176e7-44c3-4647-aba7-90365c26fb09 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:42.606850 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2167416-1b49-46b4-83c0-716956a16aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:42.607741 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2167416-1b49-46b4-83c0-716956a16aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:42.608198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2167416-1b49-46b4-83c0-716956a16aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:42.608632 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2167416-1b49-46b4-83c0-716956a16aa0 admin admin] PATH_INFO: `/v3/projects/f53ee2df7667452a87e2b377ad1b1924` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:42.611448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2167416-1b49-46b4-83c0-716956a16aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f53ee2df7667452a87e2b377ad1b1924)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:42.613987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2167416-1b49-46b4-83c0-716956a16aa0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:42.686194 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:42.779027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f2167416-1b49-46b4-83c0-716956a16aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f53ee2df7667452a87e2b377ad1b1924', 'request_id': 'req-f2167416-1b49-46b4-83c0-716956a16aa0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:42.799410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f2167416-1b49-46b4-83c0-716956a16aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f53ee2df7667452a87e2b377ad1b1924', 'request_id': 'req-f2167416-1b49-46b4-83c0-716956a16aa0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:42.905878 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f2167416-1b49-46b4-83c0-716956a16aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f53ee2df7667452a87e2b377ad1b1924', 'request_id': 'req-f2167416-1b49-46b4-83c0-716956a16aa0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:42.971569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f2167416-1b49-46b4-83c0-716956a16aa0 admin admin] The token cache is being invalidate because project f53ee2df7667452a87e2b377ad1b1924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:07:42.975418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f2167416-1b49-46b4-83c0-716956a16aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:42.993961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f2167416-1b49-46b4-83c0-716956a16aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:42.996622 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1151/2295] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:07:42 2022] DELETE /identity/v3/projects/f53ee2df7667452a87e2b377ad1b1924 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:43.002251 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a4192-a7e4-44cf-944f-4b34ef120919 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:43.017416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a4192-a7e4-44cf-944f-4b34ef120919 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:43.175747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e92b0-5995-479a-9b3e-7d90b68e29ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:43.176443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e92b0-5995-479a-9b3e-7d90b68e29ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:43.177287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e92b0-5995-479a-9b3e-7d90b68e29ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:43.180013 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e92b0-5995-479a-9b3e-7d90b68e29ea admin admin] PATH_INFO: `/v3/users/d37ae214fccb43d295d7a94ca76c30cf` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:43.181959 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e92b0-5995-479a-9b3e-7d90b68e29ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d37ae214fccb43d295d7a94ca76c30cf)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:43.184278 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e92b0-5995-479a-9b3e-7d90b68e29ea admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:43.585270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2b9e92b0-5995-479a-9b3e-7d90b68e29ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd37ae214fccb43d295d7a94ca76c30cf'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:43.587342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2b9e92b0-5995-479a-9b3e-7d90b68e29ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd37ae214fccb43d295d7a94ca76c30cf'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:43.588276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2b9e92b0-5995-479a-9b3e-7d90b68e29ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd37ae214fccb43d295d7a94ca76c30cf'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:43.597888 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2b9e92b0-5995-479a-9b3e-7d90b68e29ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd37ae214fccb43d295d7a94ca76c30cf'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:43.621437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2b9e92b0-5995-479a-9b3e-7d90b68e29ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd37ae214fccb43d295d7a94ca76c30cf'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:43.750784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1145/2296] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:07:43 2022] DELETE /identity/v3/users/d37ae214fccb43d295d7a94ca76c30cf => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:44.766575 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478cc15-3d2d-43d0-9076-a5f0b459f594 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:44.767195 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478cc15-3d2d-43d0-9076-a5f0b459f594 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:44.767414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478cc15-3d2d-43d0-9076-a5f0b459f594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:44.829649 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c478cc15-3d2d-43d0-9076-a5f0b459f594 None None] MFA Rules not processed for user `2fc5fd88556145fda1fb0067d5e9adcc`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:44.831326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c478cc15-3d2d-43d0-9076-a5f0b459f594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:44.957957 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:45.015732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e92b0-5995-479a-9b3e-7d90b68e29ea admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:45.017327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-2b9e92b0-5995-479a-9b3e-7d90b68e29ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:45.031972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1152/2297] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:45.036778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478cc15-3d2d-43d0-9076-a5f0b459f594 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:45.038618 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c478cc15-3d2d-43d0-9076-a5f0b459f594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:45.129298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:45.151959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478cc15-3d2d-43d0-9076-a5f0b459f594 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:45.160179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03835b4-19f5-4a36-a6c9-b332bd90bb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:45.161958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03835b4-19f5-4a36-a6c9-b332bd90bb93 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:45.164589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03835b4-19f5-4a36-a6c9-b332bd90bb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:45.164589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03835b4-19f5-4a36-a6c9-b332bd90bb93 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:45.165658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03835b4-19f5-4a36-a6c9-b332bd90bb93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:45.168480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03835b4-19f5-4a36-a6c9-b332bd90bb93 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:45.173486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e92b0-5995-479a-9b3e-7d90b68e29ea admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:45.212107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:45.212816 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:45.213107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:45.213398 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091 admin admin] PATH_INFO: `/v3/projects/cd88f24af1974b9c9a961e202c1b28bc` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:45.215121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd88f24af1974b9c9a961e202c1b28bc)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:45.217200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:45.318457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1153/2298] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:07:45 2022] POST /identity/v3/projects => generated 371 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:07:45.324156 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03835b4-19f5-4a36-a6c9-b332bd90bb93 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:45.414278 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03835b4-19f5-4a36-a6c9-b332bd90bb93 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:45.426927 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd88f24af1974b9c9a961e202c1b28bc', 'request_id': 'req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:45.437219 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd88f24af1974b9c9a961e202c1b28bc', 'request_id': 'req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:45.446468 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd88f24af1974b9c9a961e202c1b28bc', 'request_id': 'req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:45.539700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd22353-396f-4bbd-aefa-626551cc0b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:45.540494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd22353-396f-4bbd-aefa-626551cc0b1b admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:45.540911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd22353-396f-4bbd-aefa-626551cc0b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:45.541032 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd22353-396f-4bbd-aefa-626551cc0b1b admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:45.541812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd22353-396f-4bbd-aefa-626551cc0b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:45.544025 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd22353-396f-4bbd-aefa-626551cc0b1b admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:45.632027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091 admin admin] The token cache is being invalidate because project cd88f24af1974b9c9a961e202c1b28bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:07:45.632475 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:45.633156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e5638351-3acc-4d06-bc3d-8f1f0f7bb091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:45.636130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1146/2299] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:07:45 2022] DELETE /identity/v3/projects/cd88f24af1974b9c9a961e202c1b28bc => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:07:45.656107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a17465-1f6c-418a-a96f-d56391ba6e3b None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:45.656459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a17465-1f6c-418a-a96f-d56391ba6e3b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:45.656948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a17465-1f6c-418a-a96f-d56391ba6e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:45.678981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-75a17465-1f6c-418a-a96f-d56391ba6e3b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:45.680387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-75a17465-1f6c-418a-a96f-d56391ba6e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:45.703013 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1147/2300] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:45.714626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd6c6e-19e2-417e-8d7b-056d819495cf None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:45.714988 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd6c6e-19e2-417e-8d7b-056d819495cf None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:45.715418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd6c6e-19e2-417e-8d7b-056d819495cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:45.746000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-78cd6c6e-19e2-417e-8d7b-056d819495cf None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:45.747227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-78cd6c6e-19e2-417e-8d7b-056d819495cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:45.759631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1148/2301] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:07:45.762501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd6c6e-19e2-417e-8d7b-056d819495cf None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:45.764165 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-78cd6c6e-19e2-417e-8d7b-056d819495cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:45.765783 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1154/2302] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:07:45 2022] POST /identity/v3/users => generated 292 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:45.767671 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd22353-396f-4bbd-aefa-626551cc0b1b admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:45.784593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd6c6e-19e2-417e-8d7b-056d819495cf None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:45.800326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f699b3-40b3-4a5d-90ac-f3ee2723440c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:45.800326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f699b3-40b3-4a5d-90ac-f3ee2723440c admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:45.800681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f699b3-40b3-4a5d-90ac-f3ee2723440c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:45.800771 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f699b3-40b3-4a5d-90ac-f3ee2723440c admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:45.801455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f699b3-40b3-4a5d-90ac-f3ee2723440c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:45.803933 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f699b3-40b3-4a5d-90ac-f3ee2723440c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:45.818899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd22353-396f-4bbd-aefa-626551cc0b1b admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:45.828794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8461e9-eb17-4ec0-b027-3c09aa3c7a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:45.829666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8461e9-eb17-4ec0-b027-3c09aa3c7a47 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:45.829949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8461e9-eb17-4ec0-b027-3c09aa3c7a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:45.830102 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1149/2303] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:45.830554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8461e9-eb17-4ec0-b027-3c09aa3c7a47 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:45.832010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8461e9-eb17-4ec0-b027-3c09aa3c7a47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:07:45.832550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8461e9-eb17-4ec0-b027-3c09aa3c7a47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:45.836936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f699b3-40b3-4a5d-90ac-f3ee2723440c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:45.841191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8461e9-eb17-4ec0-b027-3c09aa3c7a47 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:45.856908 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f699b3-40b3-4a5d-90ac-f3ee2723440c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:45.862770 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1155/2304] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:07:45.862986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13a0a4-26fc-42fe-9335-bb2ae405c05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:45.863735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13a0a4-26fc-42fe-9335-bb2ae405c05c admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:45.864548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13a0a4-26fc-42fe-9335-bb2ae405c05c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:45.864548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13a0a4-26fc-42fe-9335-bb2ae405c05c admin admin] PATH_INFO: `/v3/projects/88400cb6d31b453baa2ea82f51ecc6b9/users/e49e9ffd54d747ab8d87dd7b7db9d741/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:45.868111 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8461e9-eb17-4ec0-b027-3c09aa3c7a47 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:45.884324 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13a0a4-26fc-42fe-9335-bb2ae405c05c admin admin] RBAC: Authorizing `identity:create_grant(project_id=88400cb6d31b453baa2ea82f51ecc6b9, user_id=e49e9ffd54d747ab8d87dd7b7db9d741, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:45.887259 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13a0a4-26fc-42fe-9335-bb2ae405c05c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:45.887836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8461e9-eb17-4ec0-b027-3c09aa3c7a47 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:45.903284 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfb80e-3c6b-4733-a1d9-b78ce7b213e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:45.904109 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfb80e-3c6b-4733-a1d9-b78ce7b213e3 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:45.904490 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfb80e-3c6b-4733-a1d9-b78ce7b213e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:45.904915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfb80e-3c6b-4733-a1d9-b78ce7b213e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:45.905733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfb80e-3c6b-4733-a1d9-b78ce7b213e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:45.907933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfb80e-3c6b-4733-a1d9-b78ce7b213e3 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:46.003756 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1150/2305] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:45 2022] PUT /identity/v3/projects/88400cb6d31b453baa2ea82f51ecc6b9/users/e49e9ffd54d747ab8d87dd7b7db9d741/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:46.008784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13a0a4-26fc-42fe-9335-bb2ae405c05c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:46.035060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1156/2306] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:07:45 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:07:46.039503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfb80e-3c6b-4733-a1d9-b78ce7b213e3 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:46.110357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13a0a4-26fc-42fe-9335-bb2ae405c05c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:46.142344 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfb80e-3c6b-4733-a1d9-b78ce7b213e3 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:46.149161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207165ff-0f0d-46cc-9c3b-2676494c210d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:46.149793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207165ff-0f0d-46cc-9c3b-2676494c210d admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:46.150179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207165ff-0f0d-46cc-9c3b-2676494c210d admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:46.150787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207165ff-0f0d-46cc-9c3b-2676494c210d admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:46.151510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207165ff-0f0d-46cc-9c3b-2676494c210d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:46.153557 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207165ff-0f0d-46cc-9c3b-2676494c210d admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:46.201679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadcae6-a7fb-464c-92be-1299c124664c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:46.205019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadcae6-a7fb-464c-92be-1299c124664c admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:46.205019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadcae6-a7fb-464c-92be-1299c124664c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:46.205019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadcae6-a7fb-464c-92be-1299c124664c admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:46.222330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadcae6-a7fb-464c-92be-1299c124664c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:46.222330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadcae6-a7fb-464c-92be-1299c124664c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:46.222330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1151/2307] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:46.225782 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadcae6-a7fb-464c-92be-1299c124664c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:46.238646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadcae6-a7fb-464c-92be-1299c124664c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:46.267912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d6bf9-46a2-4457-bcb8-845dd375bd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:46.270142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d6bf9-46a2-4457-bcb8-845dd375bd2e admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:46.270514 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d6bf9-46a2-4457-bcb8-845dd375bd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:46.270804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d6bf9-46a2-4457-bcb8-845dd375bd2e admin admin] PATH_INFO: `/v3/projects/88400cb6d31b453baa2ea82f51ecc6b9/users/e49e9ffd54d747ab8d87dd7b7db9d741/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:46.274520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d6bf9-46a2-4457-bcb8-845dd375bd2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=88400cb6d31b453baa2ea82f51ecc6b9, user_id=e49e9ffd54d747ab8d87dd7b7db9d741, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:46.284932 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d6bf9-46a2-4457-bcb8-845dd375bd2e admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:46.311889 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1152/2308] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:46 2022] PUT /identity/v3/projects/88400cb6d31b453baa2ea82f51ecc6b9/users/e49e9ffd54d747ab8d87dd7b7db9d741/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:46.319102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1157/2309] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:07:46 2022] POST /identity/v3/users => generated 288 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:46.325624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d6bf9-46a2-4457-bcb8-845dd375bd2e admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:46.419376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d6bf9-46a2-4457-bcb8-845dd375bd2e admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:46.424387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046ce79-d9bf-4416-95a3-9f2f62dab2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:46.425551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046ce79-d9bf-4416-95a3-9f2f62dab2bd admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:46.425551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046ce79-d9bf-4416-95a3-9f2f62dab2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:46.425764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046ce79-d9bf-4416-95a3-9f2f62dab2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:46.426372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046ce79-d9bf-4416-95a3-9f2f62dab2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:46.429332 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046ce79-d9bf-4416-95a3-9f2f62dab2bd admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:46.440147 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1153/2310] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:07:46.449235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207165ff-0f0d-46cc-9c3b-2676494c210d admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:46.461135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207165ff-0f0d-46cc-9c3b-2676494c210d admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:46.467976 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f95c1-f979-4a70-8eba-e65ab590cbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:46.468786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f95c1-f979-4a70-8eba-e65ab590cbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:46.469173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f95c1-f979-4a70-8eba-e65ab590cbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:46.469512 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f95c1-f979-4a70-8eba-e65ab590cbe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:46.470288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f95c1-f979-4a70-8eba-e65ab590cbe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:46.472516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f95c1-f979-4a70-8eba-e65ab590cbe9 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:46.488659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1158/2311] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:07:46.493437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046ce79-d9bf-4416-95a3-9f2f62dab2bd admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:46.505332 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046ce79-d9bf-4416-95a3-9f2f62dab2bd admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:46.510599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46a437-6a08-45e1-baff-02951ad1f004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:46.511285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46a437-6a08-45e1-baff-02951ad1f004 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:46.511603 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46a437-6a08-45e1-baff-02951ad1f004 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:46.520658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46a437-6a08-45e1-baff-02951ad1f004 admin admin] PATH_INFO: `/v3/projects/a439e371b75f49b1baf8f01c0595cac3/users/045fa2f2a7cb468d9edebd00619b43f2/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:46.529979 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46a437-6a08-45e1-baff-02951ad1f004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a439e371b75f49b1baf8f01c0595cac3, user_id=045fa2f2a7cb468d9edebd00619b43f2, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:46.532673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46a437-6a08-45e1-baff-02951ad1f004 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:46.709329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1154/2312] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:46 2022] PUT /identity/v3/projects/a439e371b75f49b1baf8f01c0595cac3/users/045fa2f2a7cb468d9edebd00619b43f2/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:46.720419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0660c8-3fd8-4c45-8625-8eb14316df01 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:46.720706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0660c8-3fd8-4c45-8625-8eb14316df01 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:46.720977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0660c8-3fd8-4c45-8625-8eb14316df01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:46.757573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-3f0660c8-3fd8-4c45-8625-8eb14316df01 None None] MFA Rules not processed for user `045fa2f2a7cb468d9edebd00619b43f2`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:46.758856 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3f0660c8-3fd8-4c45-8625-8eb14316df01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:46.858172 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1159/2313] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:46.867251 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46a437-6a08-45e1-baff-02951ad1f004 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:46.869031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-6f46a437-6a08-45e1-baff-02951ad1f004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:46.994129 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46a437-6a08-45e1-baff-02951ad1f004 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:47.004032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546182d1-10a3-4487-adb4-7783baf24d05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:47.005317 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546182d1-10a3-4487-adb4-7783baf24d05 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:47.005801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546182d1-10a3-4487-adb4-7783baf24d05 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:47.006132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546182d1-10a3-4487-adb4-7783baf24d05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:47.018969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546182d1-10a3-4487-adb4-7783baf24d05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:47.021096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546182d1-10a3-4487-adb4-7783baf24d05 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:47.039930 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1155/2314] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:47.420975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:47.835572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:48.206078 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0660c8-3fd8-4c45-8625-8eb14316df01 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:48.210022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3f0660c8-3fd8-4c45-8625-8eb14316df01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:48.382941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0660c8-3fd8-4c45-8625-8eb14316df01 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:48.399694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c05ad2-99e5-4b68-811d-67ea53868773 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a0728e2f6249bdb45d6f44fb15aa4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:48.400567 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c05ad2-99e5-4b68-811d-67ea53868773 service placement] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:48.401038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c05ad2-99e5-4b68-811d-67ea53868773 service placement] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:48.401353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c05ad2-99e5-4b68-811d-67ea53868773 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:48.403504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-32c05ad2-99e5-4b68-811d-67ea53868773 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:48.420153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c05ad2-99e5-4b68-811d-67ea53868773 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:48.425431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c05ad2-99e5-4b68-811d-67ea53868773 service placement] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:48.436371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1160/2315] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:48.481873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457dcf7-ecae-4019-9428-cf41fd22e062 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:48.482171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457dcf7-ecae-4019-9428-cf41fd22e062 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:48.482520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457dcf7-ecae-4019-9428-cf41fd22e062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:48.511684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-e457dcf7-ecae-4019-9428-cf41fd22e062 None None] MFA Rules not processed for user `61b3efca1989483d960b9f92d655d39e`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:48.513110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e457dcf7-ecae-4019-9428-cf41fd22e062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:48.592543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1156/2316] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:49.377173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d67d9-dd3a-47ff-bda5-e9dfd9a6136a None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:49.377555 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d67d9-dd3a-47ff-bda5-e9dfd9a6136a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:49.377555 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d67d9-dd3a-47ff-bda5-e9dfd9a6136a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:49.415053 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:49.458934 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-5b3d67d9-dd3a-47ff-bda5-e9dfd9a6136a None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:49.469006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5b3d67d9-dd3a-47ff-bda5-e9dfd9a6136a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:49.633909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1161/2317] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 15:07:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:49.753021 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457dcf7-ecae-4019-9428-cf41fd22e062 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:49.754536 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e457dcf7-ecae-4019-9428-cf41fd22e062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:49.877293 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457dcf7-ecae-4019-9428-cf41fd22e062 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:49.891127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313913fe-da7c-46b1-9ca1-1cbfcae72431 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:49.891127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313913fe-da7c-46b1-9ca1-1cbfcae72431 service swift] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:49.891127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313913fe-da7c-46b1-9ca1-1cbfcae72431 service swift] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:49.891127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313913fe-da7c-46b1-9ca1-1cbfcae72431 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:49.900552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313913fe-da7c-46b1-9ca1-1cbfcae72431 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:49.903098 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313913fe-da7c-46b1-9ca1-1cbfcae72431 service swift] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:49.923148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1157/2318] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:07:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:07:50.915767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:52.820773 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d67d9-dd3a-47ff-bda5-e9dfd9a6136a None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:53.099406 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d67d9-dd3a-47ff-bda5-e9dfd9a6136a None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:53.109433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dbd3f-06ee-4a92-bf5f-92e5b7bdad1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:53.111222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dbd3f-06ee-4a92-bf5f-92e5b7bdad1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:53.111890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dbd3f-06ee-4a92-bf5f-92e5b7bdad1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:53.112464 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dbd3f-06ee-4a92-bf5f-92e5b7bdad1c admin admin] PATH_INFO: `/v3/users/045fa2f2a7cb468d9edebd00619b43f2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:53.115026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dbd3f-06ee-4a92-bf5f-92e5b7bdad1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=045fa2f2a7cb468d9edebd00619b43f2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:53.121587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dbd3f-06ee-4a92-bf5f-92e5b7bdad1c admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:53.641193 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b69dbd3f-06ee-4a92-bf5f-92e5b7bdad1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '045fa2f2a7cb468d9edebd00619b43f2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:53.671825 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b69dbd3f-06ee-4a92-bf5f-92e5b7bdad1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '045fa2f2a7cb468d9edebd00619b43f2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:53.777784 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b69dbd3f-06ee-4a92-bf5f-92e5b7bdad1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '045fa2f2a7cb468d9edebd00619b43f2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:53.828009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:53.830165 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b69dbd3f-06ee-4a92-bf5f-92e5b7bdad1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '045fa2f2a7cb468d9edebd00619b43f2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:53.830965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b69dbd3f-06ee-4a92-bf5f-92e5b7bdad1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '045fa2f2a7cb468d9edebd00619b43f2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:53.869011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1162/2319] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:07:52 2022] DELETE /identity/v3/users/045fa2f2a7cb468d9edebd00619b43f2 => generated 0 bytes in 1046 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:53.871044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b432986-0fd4-4b4a-ab06-f0b7e7916870 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:53.872057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b432986-0fd4-4b4a-ab06-f0b7e7916870 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:53.872562 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b432986-0fd4-4b4a-ab06-f0b7e7916870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:53.899588 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-9b432986-0fd4-4b4a-ab06-f0b7e7916870 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:53.901411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9b432986-0fd4-4b4a-ab06-f0b7e7916870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:54.029106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1158/2320] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:54.098060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dbd3f-06ee-4a92-bf5f-92e5b7bdad1c admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:54.100452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b69dbd3f-06ee-4a92-bf5f-92e5b7bdad1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:54.270886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dbd3f-06ee-4a92-bf5f-92e5b7bdad1c admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:54.280579 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ff6be-4e38-4212-90bd-bcaca0fcff0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:54.281209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ff6be-4e38-4212-90bd-bcaca0fcff0e service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:54.283709 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ff6be-4e38-4212-90bd-bcaca0fcff0e service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:54.283997 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ff6be-4e38-4212-90bd-bcaca0fcff0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:54.342652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ff6be-4e38-4212-90bd-bcaca0fcff0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:54.349189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ff6be-4e38-4212-90bd-bcaca0fcff0e service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:54.392964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1163/2321] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:07:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:07:57.385220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b432986-0fd4-4b4a-ab06-f0b7e7916870 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:57.386916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9b432986-0fd4-4b4a-ab06-f0b7e7916870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:57.415633 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b432986-0fd4-4b4a-ab06-f0b7e7916870 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:57.431226 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a3505-ec58-4148-b393-af8c44d395d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:57.432871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a3505-ec58-4148-b393-af8c44d395d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:57.451008 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a3505-ec58-4148-b393-af8c44d395d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:57.451610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a3505-ec58-4148-b393-af8c44d395d3 admin admin] PATH_INFO: `/v3/projects/a439e371b75f49b1baf8f01c0595cac3` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:57.453567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a3505-ec58-4148-b393-af8c44d395d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a439e371b75f49b1baf8f01c0595cac3)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:57.456998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a3505-ec58-4148-b393-af8c44d395d3 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:57.673572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3b6a3505-ec58-4148-b393-af8c44d395d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a439e371b75f49b1baf8f01c0595cac3', 'request_id': 'req-3b6a3505-ec58-4148-b393-af8c44d395d3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:57.682806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3b6a3505-ec58-4148-b393-af8c44d395d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a439e371b75f49b1baf8f01c0595cac3', 'request_id': 'req-3b6a3505-ec58-4148-b393-af8c44d395d3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:57.690787 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3b6a3505-ec58-4148-b393-af8c44d395d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a439e371b75f49b1baf8f01c0595cac3', 'request_id': 'req-3b6a3505-ec58-4148-b393-af8c44d395d3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:57.695811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:57.817624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3b6a3505-ec58-4148-b393-af8c44d395d3 admin admin] The token cache is being invalidate because project a439e371b75f49b1baf8f01c0595cac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:07:57.817744 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3b6a3505-ec58-4148-b393-af8c44d395d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:57.822742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3b6a3505-ec58-4148-b393-af8c44d395d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:07:57.826416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1159/2322] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:07:57 2022] DELETE /identity/v3/projects/a439e371b75f49b1baf8f01c0595cac3 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:57.831439 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73443a-a586-493b-8d65-8e7442090750 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:57.832114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73443a-a586-493b-8d65-8e7442090750 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:57.832114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73443a-a586-493b-8d65-8e7442090750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:57.879245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-ce73443a-a586-493b-8d65-8e7442090750 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:57.881623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ce73443a-a586-493b-8d65-8e7442090750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:58.079844 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1164/2323] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:07:58.080518 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7eb4c2-4a85-4522-a090-db1936d7da17 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:58.080808 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7eb4c2-4a85-4522-a090-db1936d7da17 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:58.081168 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7eb4c2-4a85-4522-a090-db1936d7da17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:58.120627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-5c7eb4c2-4a85-4522-a090-db1936d7da17 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:58.121896 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-5c7eb4c2-4a85-4522-a090-db1936d7da17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:58.183307 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1160/2324] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:07:58.189121 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73443a-a586-493b-8d65-8e7442090750 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:58.228509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73443a-a586-493b-8d65-8e7442090750 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:58.258226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcd0f9-9452-426c-bac4-3f42576d87f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:58.258948 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcd0f9-9452-426c-bac4-3f42576d87f8 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:58.260586 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcd0f9-9452-426c-bac4-3f42576d87f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:58.260892 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcd0f9-9452-426c-bac4-3f42576d87f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:58.261629 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcd0f9-9452-426c-bac4-3f42576d87f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:07:58.261946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcd0f9-9452-426c-bac4-3f42576d87f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:58.276400 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcd0f9-9452-426c-bac4-3f42576d87f8 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:58.360231 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7eb4c2-4a85-4522-a090-db1936d7da17 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:58.360381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1165/2325] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:07:58.371501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7eb4c2-4a85-4522-a090-db1936d7da17 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:58.407565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa13a86-acd4-4811-842d-8f79a95ac091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:58.411262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa13a86-acd4-4811-842d-8f79a95ac091 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:58.411262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa13a86-acd4-4811-842d-8f79a95ac091 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:58.411262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa13a86-acd4-4811-842d-8f79a95ac091 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:58.411262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa13a86-acd4-4811-842d-8f79a95ac091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:58.414494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa13a86-acd4-4811-842d-8f79a95ac091 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:58.552110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1161/2326] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:07:58 2022] POST /identity/v3/projects => generated 367 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:07:58.559849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcd0f9-9452-426c-bac4-3f42576d87f8 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:58.596670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcd0f9-9452-426c-bac4-3f42576d87f8 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:58.819938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7232c2-e30b-4bcb-94d2-e6482c33800e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:58.822234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7232c2-e30b-4bcb-94d2-e6482c33800e admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:58.822234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7232c2-e30b-4bcb-94d2-e6482c33800e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:58.822234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7232c2-e30b-4bcb-94d2-e6482c33800e admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:58.822234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7232c2-e30b-4bcb-94d2-e6482c33800e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:58.826131 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7232c2-e30b-4bcb-94d2-e6482c33800e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:58.957083 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1166/2327] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:07:58 2022] POST /identity/v3/users => generated 284 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:07:58.959353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa13a86-acd4-4811-842d-8f79a95ac091 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:58.975935 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa13a86-acd4-4811-842d-8f79a95ac091 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:58.982255 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be61fd-eb02-46f5-9a61-b7604bd92d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:58.983385 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be61fd-eb02-46f5-9a61-b7604bd92d4a admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:58.984093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be61fd-eb02-46f5-9a61-b7604bd92d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:58.984670 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be61fd-eb02-46f5-9a61-b7604bd92d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:58.985627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be61fd-eb02-46f5-9a61-b7604bd92d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:58.987790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be61fd-eb02-46f5-9a61-b7604bd92d4a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:59.012451 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1162/2328] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:07:59.018895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7232c2-e30b-4bcb-94d2-e6482c33800e admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:59.033181 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7232c2-e30b-4bcb-94d2-e6482c33800e admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:59.047493 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06e145-c25e-40ac-8217-de876c4373fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:59.047493 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06e145-c25e-40ac-8217-de876c4373fe admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:59.048996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06e145-c25e-40ac-8217-de876c4373fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:59.049345 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06e145-c25e-40ac-8217-de876c4373fe admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:59.050915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06e145-c25e-40ac-8217-de876c4373fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:59.057971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06e145-c25e-40ac-8217-de876c4373fe admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:59.089611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be61fd-eb02-46f5-9a61-b7604bd92d4a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:07:59.104534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1167/2329] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:07:59.122924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be61fd-eb02-46f5-9a61-b7604bd92d4a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:07:59.128495 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c2bb1-9b1d-47c5-9201-84310fc512b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:07:59.129738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c2bb1-9b1d-47c5-9201-84310fc512b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:59.130508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c2bb1-9b1d-47c5-9201-84310fc512b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:59.131172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c2bb1-9b1d-47c5-9201-84310fc512b6 admin admin] PATH_INFO: `/v3/projects/d8b6c75aabb24313895a8a823917fc91/users/a509668af8b44c49b273d789e8a8aac3/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:59.153764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c2bb1-9b1d-47c5-9201-84310fc512b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b6c75aabb24313895a8a823917fc91, user_id=a509668af8b44c49b273d789e8a8aac3, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:07:59.157039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c2bb1-9b1d-47c5-9201-84310fc512b6 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:07:59.301327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1163/2330] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:07:59 2022] PUT /identity/v3/projects/d8b6c75aabb24313895a8a823917fc91/users/a509668af8b44c49b273d789e8a8aac3/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:07:59.311658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029f7bf-d87e-42f8-9b5d-c854ededfc7b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:07:59.311967 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029f7bf-d87e-42f8-9b5d-c854ededfc7b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:07:59.312288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029f7bf-d87e-42f8-9b5d-c854ededfc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:07:59.323716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:07:59.387153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-a029f7bf-d87e-42f8-9b5d-c854ededfc7b None None] MFA Rules not processed for user `a509668af8b44c49b273d789e8a8aac3`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:07:59.388742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a029f7bf-d87e-42f8-9b5d-c854ededfc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:07:59.548955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1168/2331] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:07:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:08:01.712672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f12b1-e3e9-406f-846d-acdc13167fe0 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:01.713164 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f12b1-e3e9-406f-846d-acdc13167fe0 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:01.713263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f12b1-e3e9-406f-846d-acdc13167fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:01.803693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-4d3f12b1-e3e9-406f-846d-acdc13167fe0 None None] MFA Rules not processed for user `e49e9ffd54d747ab8d87dd7b7db9d741`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:01.807346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4d3f12b1-e3e9-406f-846d-acdc13167fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:01.983619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1164/2332] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:08:01 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:08:02.074582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029f7bf-d87e-42f8-9b5d-c854ededfc7b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:02.076691 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a029f7bf-d87e-42f8-9b5d-c854ededfc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:02.254584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029f7bf-d87e-42f8-9b5d-c854ededfc7b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:02.267226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0304c2-20c7-40b2-bbb8-07154706a305 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:02.275039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0304c2-20c7-40b2-bbb8-07154706a305 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:02.275039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0304c2-20c7-40b2-bbb8-07154706a305 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:02.275039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0304c2-20c7-40b2-bbb8-07154706a305 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:02.275039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-fa0304c2-20c7-40b2-bbb8-07154706a305 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:02.398074 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0304c2-20c7-40b2-bbb8-07154706a305 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:02.399888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0304c2-20c7-40b2-bbb8-07154706a305 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:02.424338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1169/2333] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:08:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:08:03.981575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f12b1-e3e9-406f-846d-acdc13167fe0 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:03.984314 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4d3f12b1-e3e9-406f-846d-acdc13167fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:04.070133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f12b1-e3e9-406f-846d-acdc13167fe0 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:04.076211 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41687eb5-5b56-488d-9070-75d6a347afd2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b153935910c142aa80792e111872318a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:04.077265 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41687eb5-5b56-488d-9070-75d6a347afd2 service glance] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:04.077862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41687eb5-5b56-488d-9070-75d6a347afd2 service glance] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:04.078426 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41687eb5-5b56-488d-9070-75d6a347afd2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:04.090477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41687eb5-5b56-488d-9070-75d6a347afd2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:04.092460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41687eb5-5b56-488d-9070-75d6a347afd2 service glance] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:04.115587 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1165/2334] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:08:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:08:06.735942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adba41a-b2df-49fc-832f-3c3a71a08860 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:06.736916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adba41a-b2df-49fc-832f-3c3a71a08860 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:06.737378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adba41a-b2df-49fc-832f-3c3a71a08860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:06.769848 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-6adba41a-b2df-49fc-832f-3c3a71a08860 None None] MFA Rules not processed for user `2fc5fd88556145fda1fb0067d5e9adcc`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:06.771145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6adba41a-b2df-49fc-832f-3c3a71a08860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:06.780096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1170/2335] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:08:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:08:14.099480 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:DELETE=3,keystone:SELECT=91,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:16.763282 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:30.184271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efe2bb-fede-4c30-9953-e1559c52fe48 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:30.184824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efe2bb-fede-4c30-9953-e1559c52fe48 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:30.184824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efe2bb-fede-4c30-9953-e1559c52fe48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:30.205608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-42efe2bb-fede-4c30-9953-e1559c52fe48 None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:30.207867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-42efe2bb-fede-4c30-9953-e1559c52fe48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:30.271334 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1166/2336] 10.222.0.94 () {58 vars in 967 bytes} [Tue Jul 26 15:08:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:08:30.287538 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adba41a-b2df-49fc-832f-3c3a71a08860 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:30.289109 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6adba41a-b2df-49fc-832f-3c3a71a08860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:30.365932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adba41a-b2df-49fc-832f-3c3a71a08860 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:30.371221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9c27c-2e1d-4f8c-84a8-e8ded2ddca15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:30.371868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9c27c-2e1d-4f8c-84a8-e8ded2ddca15 service swift] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:30.372143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9c27c-2e1d-4f8c-84a8-e8ded2ddca15 service swift] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:30.372412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9c27c-2e1d-4f8c-84a8-e8ded2ddca15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:30.382003 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9c27c-2e1d-4f8c-84a8-e8ded2ddca15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:30.383609 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9c27c-2e1d-4f8c-84a8-e8ded2ddca15 service swift] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:30.392580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1171/2337] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:08:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 15:08:35.548571 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083bda1-8e93-4dd4-9106-7f7c7a3bb808 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:35.549247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083bda1-8e93-4dd4-9106-7f7c7a3bb808 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:35.549247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083bda1-8e93-4dd4-9106-7f7c7a3bb808 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:35.551448 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1167/2338] 10.222.0.94 () {54 vars in 921 bytes} [Tue Jul 26 15:08:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 15:08:35.579734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d2a78-725f-4fd0-b1c9-8638bf224d8b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:35.580630 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d2a78-725f-4fd0-b1c9-8638bf224d8b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:35.580937 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d2a78-725f-4fd0-b1c9-8638bf224d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:35.603690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-804d2a78-725f-4fd0-b1c9-8638bf224d8b None None] MFA Rules not processed for user `42a0728e2f6249bdb45d6f44fb15aa4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:35.605054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-804d2a78-725f-4fd0-b1c9-8638bf224d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:35.681247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1172/2339] 10.222.0.94 () {58 vars in 1018 bytes} [Tue Jul 26 15:08:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:08:35.694345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083bda1-8e93-4dd4-9106-7f7c7a3bb808 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:35.703034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-7083bda1-8e93-4dd4-9106-7f7c7a3bb808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:35.703254 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d2a78-725f-4fd0-b1c9-8638bf224d8b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:35.703878 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-804d2a78-725f-4fd0-b1c9-8638bf224d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:35.742336 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083bda1-8e93-4dd4-9106-7f7c7a3bb808 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:35.754057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8822826-ab71-49b1-8fd1-928b461f5496 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a0728e2f6249bdb45d6f44fb15aa4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:35.754057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822826-ab71-49b1-8fd1-928b461f5496 service placement] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:35.754057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822826-ab71-49b1-8fd1-928b461f5496 service placement] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:35.754057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822826-ab71-49b1-8fd1-928b461f5496 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:35.760300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8822826-ab71-49b1-8fd1-928b461f5496 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:35.762221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8822826-ab71-49b1-8fd1-928b461f5496 service placement] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:35.777629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1168/2340] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:08:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:08:35.794470 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d2a78-725f-4fd0-b1c9-8638bf224d8b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:35.798407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770c2d8-dd95-4911-883b-ba7f4bfe71eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:35.799360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770c2d8-dd95-4911-883b-ba7f4bfe71eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:35.800703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770c2d8-dd95-4911-883b-ba7f4bfe71eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:35.800703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770c2d8-dd95-4911-883b-ba7f4bfe71eb admin admin] PATH_INFO: `/v3/users/2bf28a5c061c42668fc54a0c50af64ee` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:35.801801 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770c2d8-dd95-4911-883b-ba7f4bfe71eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf28a5c061c42668fc54a0c50af64ee)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:35.803894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770c2d8-dd95-4911-883b-ba7f4bfe71eb admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:35.949637 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:36.140911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5770c2d8-dd95-4911-883b-ba7f4bfe71eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf28a5c061c42668fc54a0c50af64ee'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:36.148179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5770c2d8-dd95-4911-883b-ba7f4bfe71eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf28a5c061c42668fc54a0c50af64ee'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:36.303348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5770c2d8-dd95-4911-883b-ba7f4bfe71eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf28a5c061c42668fc54a0c50af64ee'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:36.317433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5770c2d8-dd95-4911-883b-ba7f4bfe71eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf28a5c061c42668fc54a0c50af64ee'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:36.318172 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5770c2d8-dd95-4911-883b-ba7f4bfe71eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf28a5c061c42668fc54a0c50af64ee'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:36.324251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1173/2341] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:08:35 2022] DELETE /identity/v3/users/2bf28a5c061c42668fc54a0c50af64ee => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:08:36.607312 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8822826-ab71-49b1-8fd1-928b461f5496 service placement] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:36.620919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8822826-ab71-49b1-8fd1-928b461f5496 service placement] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:36.686332 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac54ed-fa03-4c70-bafb-9627d7760280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:36.687023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac54ed-fa03-4c70-bafb-9627d7760280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:36.687400 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac54ed-fa03-4c70-bafb-9627d7760280 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:36.687817 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac54ed-fa03-4c70-bafb-9627d7760280 admin admin] PATH_INFO: `/v3/projects/4272758d01274f419bc75c85140daa06` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:36.689498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac54ed-fa03-4c70-bafb-9627d7760280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4272758d01274f419bc75c85140daa06)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:36.692440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac54ed-fa03-4c70-bafb-9627d7760280 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:36.830037 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2fac54ed-fa03-4c70-bafb-9627d7760280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4272758d01274f419bc75c85140daa06', 'request_id': 'req-2fac54ed-fa03-4c70-bafb-9627d7760280'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:36.837367 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2fac54ed-fa03-4c70-bafb-9627d7760280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4272758d01274f419bc75c85140daa06', 'request_id': 'req-2fac54ed-fa03-4c70-bafb-9627d7760280'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:36.843273 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2fac54ed-fa03-4c70-bafb-9627d7760280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4272758d01274f419bc75c85140daa06', 'request_id': 'req-2fac54ed-fa03-4c70-bafb-9627d7760280'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:37.005259 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2fac54ed-fa03-4c70-bafb-9627d7760280 admin admin] The token cache is being invalidate because project 4272758d01274f419bc75c85140daa06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:08:37.005573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2fac54ed-fa03-4c70-bafb-9627d7760280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:37.006227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-2fac54ed-fa03-4c70-bafb-9627d7760280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:37.007944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1169/2342] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:08:36 2022] DELETE /identity/v3/projects/4272758d01274f419bc75c85140daa06 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:08:37.013050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693088a-6fb4-4eac-95ab-26ca1855e5d7 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:37.013524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693088a-6fb4-4eac-95ab-26ca1855e5d7 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:37.013648 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693088a-6fb4-4eac-95ab-26ca1855e5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:37.033658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b693088a-6fb4-4eac-95ab-26ca1855e5d7 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:37.034900 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b693088a-6fb4-4eac-95ab-26ca1855e5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:37.101929 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1174/2343] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:08:37.116286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59784639-32af-4044-9f54-f3bc09099a10 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:37.116573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59784639-32af-4044-9f54-f3bc09099a10 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:37.116856 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59784639-32af-4044-9f54-f3bc09099a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:37.141432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-59784639-32af-4044-9f54-f3bc09099a10 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:37.142637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-59784639-32af-4044-9f54-f3bc09099a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:37.151386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1170/2344] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:08:37.156536 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693088a-6fb4-4eac-95ab-26ca1855e5d7 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:37.167702 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693088a-6fb4-4eac-95ab-26ca1855e5d7 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:37.173182 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a97e3-8efe-4896-8ebd-8fcbde5f7245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:37.173855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a97e3-8efe-4896-8ebd-8fcbde5f7245 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:37.174138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a97e3-8efe-4896-8ebd-8fcbde5f7245 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:37.174912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a97e3-8efe-4896-8ebd-8fcbde5f7245 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:37.176674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a97e3-8efe-4896-8ebd-8fcbde5f7245 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:08:37.176674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a97e3-8efe-4896-8ebd-8fcbde5f7245 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:37.177967 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a97e3-8efe-4896-8ebd-8fcbde5f7245 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:37.194891 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1175/2345] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:08:37.200051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59784639-32af-4044-9f54-f3bc09099a10 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:37.210737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59784639-32af-4044-9f54-f3bc09099a10 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:37.215714 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5de85-1c05-4331-bcd8-838ed6afb7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:37.216337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5de85-1c05-4331-bcd8-838ed6afb7da admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:37.216623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5de85-1c05-4331-bcd8-838ed6afb7da admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:37.216905 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5de85-1c05-4331-bcd8-838ed6afb7da admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:37.217597 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5de85-1c05-4331-bcd8-838ed6afb7da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:37.219673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5de85-1c05-4331-bcd8-838ed6afb7da admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:37.237858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:37.345679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1171/2346] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:08:37 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:08:37.350704 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a97e3-8efe-4896-8ebd-8fcbde5f7245 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:37.361033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a97e3-8efe-4896-8ebd-8fcbde5f7245 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:37.427297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c7d1a-7388-4708-9cf8-c308eae65bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:37.427956 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c7d1a-7388-4708-9cf8-c308eae65bad admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:37.428236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c7d1a-7388-4708-9cf8-c308eae65bad admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:37.428530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c7d1a-7388-4708-9cf8-c308eae65bad admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:37.429192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c7d1a-7388-4708-9cf8-c308eae65bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:37.431102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c7d1a-7388-4708-9cf8-c308eae65bad admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:37.439221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5de85-1c05-4331-bcd8-838ed6afb7da admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:37.441674 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-09b5de85-1c05-4331-bcd8-838ed6afb7da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:37.454991 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5de85-1c05-4331-bcd8-838ed6afb7da admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:37.460434 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d925d-71dd-4c76-8b04-29bcecd635df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:37.461071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d925d-71dd-4c76-8b04-29bcecd635df admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:37.461353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d925d-71dd-4c76-8b04-29bcecd635df admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:37.461626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d925d-71dd-4c76-8b04-29bcecd635df admin admin] PATH_INFO: `/v3/users/c6984944b2ee4f6bb958a2f4a9f08172` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:37.462793 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d925d-71dd-4c76-8b04-29bcecd635df admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6984944b2ee4f6bb958a2f4a9f08172)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:37.464733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d925d-71dd-4c76-8b04-29bcecd635df admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:37.611793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1176/2347] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:08:37 2022] POST /identity/v3/users => generated 287 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:08:37.621792 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c7d1a-7388-4708-9cf8-c308eae65bad admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:37.635068 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c7d1a-7388-4708-9cf8-c308eae65bad admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:37.639985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4295da4-9aa6-475a-8b50-9604e36e8aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:37.640610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4295da4-9aa6-475a-8b50-9604e36e8aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:37.640885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4295da4-9aa6-475a-8b50-9604e36e8aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:37.641186 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4295da4-9aa6-475a-8b50-9604e36e8aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:37.641882 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4295da4-9aa6-475a-8b50-9604e36e8aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:37.644503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4295da4-9aa6-475a-8b50-9604e36e8aa3 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:37.658812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1177/2348] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:08:37.663690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4295da4-9aa6-475a-8b50-9604e36e8aa3 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:37.674560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4295da4-9aa6-475a-8b50-9604e36e8aa3 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:37.679760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549f030-a26e-4b59-8d40-552e65ca858f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:37.680391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549f030-a26e-4b59-8d40-552e65ca858f admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:37.680673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549f030-a26e-4b59-8d40-552e65ca858f admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:37.680946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549f030-a26e-4b59-8d40-552e65ca858f admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:37.681651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549f030-a26e-4b59-8d40-552e65ca858f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:37.683240 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549f030-a26e-4b59-8d40-552e65ca858f admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:37.689084 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:37.695003 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1178/2349] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:08:37.699715 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549f030-a26e-4b59-8d40-552e65ca858f admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:37.712035 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549f030-a26e-4b59-8d40-552e65ca858f admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:37.716924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f862f-10a9-4a0b-aff6-c50099be0685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:37.717608 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f862f-10a9-4a0b-aff6-c50099be0685 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:37.717883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f862f-10a9-4a0b-aff6-c50099be0685 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:37.718280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f862f-10a9-4a0b-aff6-c50099be0685 admin admin] PATH_INFO: `/v3/projects/dc65e563bd7e49f6a94fb73175cf0519/users/58d6daedfa8d4ee1a714daa2da2964c2/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:37.738398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f862f-10a9-4a0b-aff6-c50099be0685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc65e563bd7e49f6a94fb73175cf0519, user_id=58d6daedfa8d4ee1a714daa2da2964c2, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:37.741382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f862f-10a9-4a0b-aff6-c50099be0685 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:37.769854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-117d925d-71dd-4c76-8b04-29bcecd635df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6984944b2ee4f6bb958a2f4a9f08172'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:37.770522 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-117d925d-71dd-4c76-8b04-29bcecd635df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6984944b2ee4f6bb958a2f4a9f08172'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:37.771154 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-117d925d-71dd-4c76-8b04-29bcecd635df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6984944b2ee4f6bb958a2f4a9f08172'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:37.779962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-117d925d-71dd-4c76-8b04-29bcecd635df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6984944b2ee4f6bb958a2f4a9f08172'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:37.795825 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-117d925d-71dd-4c76-8b04-29bcecd635df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6984944b2ee4f6bb958a2f4a9f08172'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:37.840997 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1179/2350] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:08:37 2022] PUT /identity/v3/projects/dc65e563bd7e49f6a94fb73175cf0519/users/58d6daedfa8d4ee1a714daa2da2964c2/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:08:37.913302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d3d87-82dc-4c07-8eaa-da20f500d381 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:37.913597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d3d87-82dc-4c07-8eaa-da20f500d381 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:37.913842 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d3d87-82dc-4c07-8eaa-da20f500d381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:37.917918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1172/2351] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:08:37 2022] DELETE /identity/v3/users/c6984944b2ee4f6bb958a2f4a9f08172 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:08:37.923165 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc688ba-7336-4075-8279-0aecbb4b6c15 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:37.923478 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc688ba-7336-4075-8279-0aecbb4b6c15 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:37.923769 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc688ba-7336-4075-8279-0aecbb4b6c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:37.944244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-6dc688ba-7336-4075-8279-0aecbb4b6c15 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:37.945644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-6dc688ba-7336-4075-8279-0aecbb4b6c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:37.945769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-3f0d3d87-82dc-4c07-8eaa-da20f500d381 None None] MFA Rules not processed for user `58d6daedfa8d4ee1a714daa2da2964c2`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:37.947130 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3f0d3d87-82dc-4c07-8eaa-da20f500d381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:38.009750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1173/2352] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:08:38.020315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc688ba-7336-4075-8279-0aecbb4b6c15 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:38.021775 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-6dc688ba-7336-4075-8279-0aecbb4b6c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:38.024767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1180/2353] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:08:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:08:38.035924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d3d87-82dc-4c07-8eaa-da20f500d381 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:38.037486 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3f0d3d87-82dc-4c07-8eaa-da20f500d381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:38.091803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc688ba-7336-4075-8279-0aecbb4b6c15 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:38.095652 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122287d-fe54-47bc-8f1d-668dd2cab0f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:38.096907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122287d-fe54-47bc-8f1d-668dd2cab0f3 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:38.097204 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122287d-fe54-47bc-8f1d-668dd2cab0f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:38.097483 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122287d-fe54-47bc-8f1d-668dd2cab0f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:38.099963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d3d87-82dc-4c07-8eaa-da20f500d381 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:38.103658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e10b6-c6d7-4d06-9aef-fe25a1b79228 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:38.104278 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e10b6-c6d7-4d06-9aef-fe25a1b79228 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:38.105141 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e10b6-c6d7-4d06-9aef-fe25a1b79228 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:38.105141 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e10b6-c6d7-4d06-9aef-fe25a1b79228 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:38.107092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122287d-fe54-47bc-8f1d-668dd2cab0f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:38.108499 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122287d-fe54-47bc-8f1d-668dd2cab0f3 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:38.117212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e10b6-c6d7-4d06-9aef-fe25a1b79228 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:38.119034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e10b6-c6d7-4d06-9aef-fe25a1b79228 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:38.119177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1174/2354] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:08:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:08:38.134286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1181/2355] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:08:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:08:38.624193 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122287d-fe54-47bc-8f1d-668dd2cab0f3 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:38.625944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a122287d-fe54-47bc-8f1d-668dd2cab0f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:38.658572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122287d-fe54-47bc-8f1d-668dd2cab0f3 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:38.658572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd66f0-976c-4b56-a119-732fe8d482ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:38.658572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd66f0-976c-4b56-a119-732fe8d482ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:38.658572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd66f0-976c-4b56-a119-732fe8d482ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:38.659982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd66f0-976c-4b56-a119-732fe8d482ab admin admin] PATH_INFO: `/v3/projects/e9eba5436c0c4f52b5c4e7412176a751` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:38.659982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd66f0-976c-4b56-a119-732fe8d482ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9eba5436c0c4f52b5c4e7412176a751)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:38.659982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd66f0-976c-4b56-a119-732fe8d482ab admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:38.659982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:38.818619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a0bd66f0-976c-4b56-a119-732fe8d482ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9eba5436c0c4f52b5c4e7412176a751', 'request_id': 'req-a0bd66f0-976c-4b56-a119-732fe8d482ab'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:38.825288 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a0bd66f0-976c-4b56-a119-732fe8d482ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9eba5436c0c4f52b5c4e7412176a751', 'request_id': 'req-a0bd66f0-976c-4b56-a119-732fe8d482ab'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:38.833189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a0bd66f0-976c-4b56-a119-732fe8d482ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9eba5436c0c4f52b5c4e7412176a751', 'request_id': 'req-a0bd66f0-976c-4b56-a119-732fe8d482ab'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:38.937171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a0bd66f0-976c-4b56-a119-732fe8d482ab admin admin] The token cache is being invalidate because project e9eba5436c0c4f52b5c4e7412176a751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:08:38.937549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a0bd66f0-976c-4b56-a119-732fe8d482ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:38.938116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a0bd66f0-976c-4b56-a119-732fe8d482ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:08:38.939919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1175/2356] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:08:38 2022] DELETE /identity/v3/projects/e9eba5436c0c4f52b5c4e7412176a751 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:08:38.946518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8c7e4-eefd-4605-aafb-6929b35dd21f None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:38.947041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8c7e4-eefd-4605-aafb-6929b35dd21f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:38.947545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8c7e4-eefd-4605-aafb-6929b35dd21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:38.970449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-3ca8c7e4-eefd-4605-aafb-6929b35dd21f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:38.971822 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3ca8c7e4-eefd-4605-aafb-6929b35dd21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:39.038566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1182/2357] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:08:39.046286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a6a2d-a2a4-4fd0-a9c1-8ce6c8338b48 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:39.046581 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a6a2d-a2a4-4fd0-a9c1-8ce6c8338b48 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:39.046888 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a6a2d-a2a4-4fd0-a9c1-8ce6c8338b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:39.070819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-495a6a2d-a2a4-4fd0-a9c1-8ce6c8338b48 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:39.072121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-495a6a2d-a2a4-4fd0-a9c1-8ce6c8338b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:39.082075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1176/2358] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:08:39.086668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8c7e4-eefd-4605-aafb-6929b35dd21f None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:39.099456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8c7e4-eefd-4605-aafb-6929b35dd21f None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:39.104540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6fe2e-d131-4cb9-a562-f104db21c62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:39.105172 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6fe2e-d131-4cb9-a562-f104db21c62d admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:39.105457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6fe2e-d131-4cb9-a562-f104db21c62d admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:39.105734 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6fe2e-d131-4cb9-a562-f104db21c62d admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:39.106639 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6fe2e-d131-4cb9-a562-f104db21c62d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:08:39.106957 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6fe2e-d131-4cb9-a562-f104db21c62d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:39.108623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6fe2e-d131-4cb9-a562-f104db21c62d admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:39.122751 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:39.123047 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1183/2359] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:08:39.127098 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a6a2d-a2a4-4fd0-a9c1-8ce6c8338b48 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:39.137171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a6a2d-a2a4-4fd0-a9c1-8ce6c8338b48 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:39.142167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c598b-6cb9-4f61-ab10-869d1bbb8f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:39.142805 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c598b-6cb9-4f61-ab10-869d1bbb8f93 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:39.143217 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c598b-6cb9-4f61-ab10-869d1bbb8f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:39.144255 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c598b-6cb9-4f61-ab10-869d1bbb8f93 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:39.145185 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c598b-6cb9-4f61-ab10-869d1bbb8f93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:39.147086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c598b-6cb9-4f61-ab10-869d1bbb8f93 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:39.259766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1177/2360] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:08:39 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:08:39.261029 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6fe2e-d131-4cb9-a562-f104db21c62d admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:39.276779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6fe2e-d131-4cb9-a562-f104db21c62d admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:39.342533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dacc63-802d-49b4-8934-73c3066d645a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:39.343066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dacc63-802d-49b4-8934-73c3066d645a admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:39.343342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dacc63-802d-49b4-8934-73c3066d645a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:39.343651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dacc63-802d-49b4-8934-73c3066d645a admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:39.344346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dacc63-802d-49b4-8934-73c3066d645a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:39.346315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dacc63-802d-49b4-8934-73c3066d645a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:39.497801 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1184/2361] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:08:39 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:08:39.499376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c598b-6cb9-4f61-ab10-869d1bbb8f93 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:39.511247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c598b-6cb9-4f61-ab10-869d1bbb8f93 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:39.516102 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94388c48-b511-4647-9eea-45cc372aca04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:39.516725 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94388c48-b511-4647-9eea-45cc372aca04 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:39.517023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94388c48-b511-4647-9eea-45cc372aca04 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:39.517267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94388c48-b511-4647-9eea-45cc372aca04 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:39.517970 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94388c48-b511-4647-9eea-45cc372aca04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:39.520154 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94388c48-b511-4647-9eea-45cc372aca04 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:39.531104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1178/2362] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:08:39.535577 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dacc63-802d-49b4-8934-73c3066d645a admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:39.546917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dacc63-802d-49b4-8934-73c3066d645a admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:39.552928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23cb4ec-5db8-4f8e-9599-6156267d08cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:39.554288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cb4ec-5db8-4f8e-9599-6156267d08cf admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:39.554288 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cb4ec-5db8-4f8e-9599-6156267d08cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:39.554491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cb4ec-5db8-4f8e-9599-6156267d08cf admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:39.555170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23cb4ec-5db8-4f8e-9599-6156267d08cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:39.557010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23cb4ec-5db8-4f8e-9599-6156267d08cf admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:39.571392 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1185/2363] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:08:39.576199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94388c48-b511-4647-9eea-45cc372aca04 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:39.588841 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94388c48-b511-4647-9eea-45cc372aca04 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:39.594157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99317a09-f457-438b-ab2a-a00104461477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:39.594768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99317a09-f457-438b-ab2a-a00104461477 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:39.595078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99317a09-f457-438b-ab2a-a00104461477 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:39.595355 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99317a09-f457-438b-ab2a-a00104461477 admin admin] PATH_INFO: `/v3/projects/484282e33cd64579b59888ba08ef08c5/users/aafb57f34153410783daed91152fa43b/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:39.613458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99317a09-f457-438b-ab2a-a00104461477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=484282e33cd64579b59888ba08ef08c5, user_id=aafb57f34153410783daed91152fa43b, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:39.616547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99317a09-f457-438b-ab2a-a00104461477 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:39.709461 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1179/2364] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:08:39 2022] PUT /identity/v3/projects/484282e33cd64579b59888ba08ef08c5/users/aafb57f34153410783daed91152fa43b/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:08:39.720532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345ee3f-a8a3-4abc-bd2b-6e5775df711b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:39.720806 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345ee3f-a8a3-4abc-bd2b-6e5775df711b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:39.721078 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345ee3f-a8a3-4abc-bd2b-6e5775df711b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:39.755524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-9345ee3f-a8a3-4abc-bd2b-6e5775df711b None None] MFA Rules not processed for user `aafb57f34153410783daed91152fa43b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:39.756936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9345ee3f-a8a3-4abc-bd2b-6e5775df711b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:39.854817 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1186/2365] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:08:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:08:39.874198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99317a09-f457-438b-ab2a-a00104461477 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:08:39.874198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-99317a09-f457-438b-ab2a-a00104461477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:39.939698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99317a09-f457-438b-ab2a-a00104461477 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:08:39.944286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cafd5-84c0-4117-a162-2714e89f2f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:08:39.945069 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cafd5-84c0-4117-a162-2714e89f2f71 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:39.945373 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cafd5-84c0-4117-a162-2714e89f2f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:39.945635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cafd5-84c0-4117-a162-2714e89f2f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:39.959083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cafd5-84c0-4117-a162-2714e89f2f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:08:39.961925 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cafd5-84c0-4117-a162-2714e89f2f71 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:08:39.970643 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1180/2366] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:08:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:08:46.238793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fd0e1-da18-438c-9ba7-a01ded100329 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:08:46.239391 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fd0e1-da18-438c-9ba7-a01ded100329 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:08:46.239616 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fd0e1-da18-438c-9ba7-a01ded100329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:08:46.267858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-5c2fd0e1-da18-438c-9ba7-a01ded100329 None None] MFA Rules not processed for user `aafb57f34153410783daed91152fa43b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:08:46.275846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5c2fd0e1-da18-438c-9ba7-a01ded100329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:08:46.293008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1187/2367] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:08:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:08:49.966082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:08:56.259129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:01.190506 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cafd5-84c0-4117-a162-2714e89f2f71 service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:01.191961 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-139cafd5-84c0-4117-a162-2714e89f2f71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:01.271092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cafd5-84c0-4117-a162-2714e89f2f71 service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:01.276410 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd5b02-23ae-4a3b-bef5-fcecde294356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:01.277084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd5b02-23ae-4a3b-bef5-fcecde294356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:01.277366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd5b02-23ae-4a3b-bef5-fcecde294356 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:01.277642 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd5b02-23ae-4a3b-bef5-fcecde294356 admin admin] PATH_INFO: `/v3/users/58d6daedfa8d4ee1a714daa2da2964c2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:01.278888 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd5b02-23ae-4a3b-bef5-fcecde294356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58d6daedfa8d4ee1a714daa2da2964c2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:01.281088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd5b02-23ae-4a3b-bef5-fcecde294356 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:01.537405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-41bd5b02-23ae-4a3b-bef5-fcecde294356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58d6daedfa8d4ee1a714daa2da2964c2'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:01.538240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-41bd5b02-23ae-4a3b-bef5-fcecde294356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58d6daedfa8d4ee1a714daa2da2964c2'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:01.539024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-41bd5b02-23ae-4a3b-bef5-fcecde294356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58d6daedfa8d4ee1a714daa2da2964c2'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:01.547289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-41bd5b02-23ae-4a3b-bef5-fcecde294356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58d6daedfa8d4ee1a714daa2da2964c2'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:01.563272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-41bd5b02-23ae-4a3b-bef5-fcecde294356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58d6daedfa8d4ee1a714daa2da2964c2'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:01.722518 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1181/2368] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:09:01 2022] DELETE /identity/v3/users/58d6daedfa8d4ee1a714daa2da2964c2 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:09:01.727433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a7fc6-f847-477c-8ac4-d7f69a09e4a6 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:01.727966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a7fc6-f847-477c-8ac4-d7f69a09e4a6 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:01.728151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a7fc6-f847-477c-8ac4-d7f69a09e4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:01.748676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-0b2a7fc6-f847-477c-8ac4-d7f69a09e4a6 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:01.749978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-0b2a7fc6-f847-477c-8ac4-d7f69a09e4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:01.838898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1188/2369] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:01.849989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd5b02-23ae-4a3b-bef5-fcecde294356 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:01.851414 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-41bd5b02-23ae-4a3b-bef5-fcecde294356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:01.923028 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd5b02-23ae-4a3b-bef5-fcecde294356 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:01.927232 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78795493-61f0-44d8-8e7b-c5b62c4f92e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:01.927912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78795493-61f0-44d8-8e7b-c5b62c4f92e9 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:01.928291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78795493-61f0-44d8-8e7b-c5b62c4f92e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:01.928473 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78795493-61f0-44d8-8e7b-c5b62c4f92e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:01.957425 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78795493-61f0-44d8-8e7b-c5b62c4f92e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:01.960003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78795493-61f0-44d8-8e7b-c5b62c4f92e9 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:01.982012 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1182/2370] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:09:03.029777 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a7fc6-f847-477c-8ac4-d7f69a09e4a6 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:03.031226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-0b2a7fc6-f847-477c-8ac4-d7f69a09e4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:03.044452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a7fc6-f847-477c-8ac4-d7f69a09e4a6 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:03.050120 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1332a90-c338-46b7-92ea-495f10690d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:03.050772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1332a90-c338-46b7-92ea-495f10690d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:03.051056 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1332a90-c338-46b7-92ea-495f10690d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:03.051328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1332a90-c338-46b7-92ea-495f10690d3d admin admin] PATH_INFO: `/v3/projects/dc65e563bd7e49f6a94fb73175cf0519` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:03.052561 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1332a90-c338-46b7-92ea-495f10690d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc65e563bd7e49f6a94fb73175cf0519)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:03.054542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1332a90-c338-46b7-92ea-495f10690d3d admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:03.202677 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b1332a90-c338-46b7-92ea-495f10690d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc65e563bd7e49f6a94fb73175cf0519', 'request_id': 'req-b1332a90-c338-46b7-92ea-495f10690d3d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:03.211786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b1332a90-c338-46b7-92ea-495f10690d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc65e563bd7e49f6a94fb73175cf0519', 'request_id': 'req-b1332a90-c338-46b7-92ea-495f10690d3d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:03.348418 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b1332a90-c338-46b7-92ea-495f10690d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc65e563bd7e49f6a94fb73175cf0519', 'request_id': 'req-b1332a90-c338-46b7-92ea-495f10690d3d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:03.395233 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b1332a90-c338-46b7-92ea-495f10690d3d admin admin] The token cache is being invalidate because project dc65e563bd7e49f6a94fb73175cf0519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:09:03.395616 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b1332a90-c338-46b7-92ea-495f10690d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:03.397031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b1332a90-c338-46b7-92ea-495f10690d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:03.398692 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1189/2371] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:09:03 2022] DELETE /identity/v3/projects/dc65e563bd7e49f6a94fb73175cf0519 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:03.403925 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a989c65-cbde-4152-8898-a144c9bc1f9e None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:03.411613 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a989c65-cbde-4152-8898-a144c9bc1f9e None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:03.411613 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a989c65-cbde-4152-8898-a144c9bc1f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:03.455089 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-4a989c65-cbde-4152-8898-a144c9bc1f9e None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:03.455089 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4a989c65-cbde-4152-8898-a144c9bc1f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:03.483204 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:03.550326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1183/2372] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:03.565927 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099e778-0432-477c-b66f-6ad2a79fee7f None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:03.566223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099e778-0432-477c-b66f-6ad2a79fee7f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:03.566502 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099e778-0432-477c-b66f-6ad2a79fee7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:03.608849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-c099e778-0432-477c-b66f-6ad2a79fee7f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:03.610091 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c099e778-0432-477c-b66f-6ad2a79fee7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:03.619159 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1190/2373] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:03.623542 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a989c65-cbde-4152-8898-a144c9bc1f9e None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:03.633985 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a989c65-cbde-4152-8898-a144c9bc1f9e None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:03.638867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7c8ec-b262-4cb3-a34a-d25caca0c4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:03.639452 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7c8ec-b262-4cb3-a34a-d25caca0c4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:03.639759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7c8ec-b262-4cb3-a34a-d25caca0c4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:03.640156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7c8ec-b262-4cb3-a34a-d25caca0c4a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:03.640742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7c8ec-b262-4cb3-a34a-d25caca0c4a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:09:03.641047 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7c8ec-b262-4cb3-a34a-d25caca0c4a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:03.642677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7c8ec-b262-4cb3-a34a-d25caca0c4a7 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:03.655450 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1184/2374] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:09:03.661323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099e778-0432-477c-b66f-6ad2a79fee7f None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:03.670906 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099e778-0432-477c-b66f-6ad2a79fee7f None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:03.676544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e22eda-38f1-482f-8a26-5f2bc91d37a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:03.677157 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e22eda-38f1-482f-8a26-5f2bc91d37a0 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:03.677443 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e22eda-38f1-482f-8a26-5f2bc91d37a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:03.677764 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e22eda-38f1-482f-8a26-5f2bc91d37a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:03.678449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e22eda-38f1-482f-8a26-5f2bc91d37a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:03.680409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e22eda-38f1-482f-8a26-5f2bc91d37a0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:03.841029 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1191/2375] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:09:03 2022] POST /identity/v3/projects => generated 381 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:03.841616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7c8ec-b262-4cb3-a34a-d25caca0c4a7 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:03.857713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7c8ec-b262-4cb3-a34a-d25caca0c4a7 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:03.966011 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec18fb4-5b33-4bdd-926e-3ff060719633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:03.966846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec18fb4-5b33-4bdd-926e-3ff060719633 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:03.967239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec18fb4-5b33-4bdd-926e-3ff060719633 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:03.967416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec18fb4-5b33-4bdd-926e-3ff060719633 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:03.968174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec18fb4-5b33-4bdd-926e-3ff060719633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:03.972679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec18fb4-5b33-4bdd-926e-3ff060719633 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:04.090131 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1185/2376] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:09:03 2022] POST /identity/v3/users => generated 291 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:04.090705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e22eda-38f1-482f-8a26-5f2bc91d37a0 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:04.107227 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e22eda-38f1-482f-8a26-5f2bc91d37a0 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:04.113364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c37852-7da3-443d-bc87-3e494d09b8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:04.114283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c37852-7da3-443d-bc87-3e494d09b8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:04.114566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c37852-7da3-443d-bc87-3e494d09b8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:04.114842 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c37852-7da3-443d-bc87-3e494d09b8c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:04.115601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c37852-7da3-443d-bc87-3e494d09b8c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:04.117541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c37852-7da3-443d-bc87-3e494d09b8c0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:04.131621 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1192/2377] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:04.135358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec18fb4-5b33-4bdd-926e-3ff060719633 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:04.152531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec18fb4-5b33-4bdd-926e-3ff060719633 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:04.161143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c56fbe-17af-44e2-895c-c253ea5ad879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:04.161791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c56fbe-17af-44e2-895c-c253ea5ad879 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:04.162080 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c56fbe-17af-44e2-895c-c253ea5ad879 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:04.162361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c56fbe-17af-44e2-895c-c253ea5ad879 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:04.168252 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c56fbe-17af-44e2-895c-c253ea5ad879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:04.176986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c56fbe-17af-44e2-895c-c253ea5ad879 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:04.195986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1186/2378] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:04.197505 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c37852-7da3-443d-bc87-3e494d09b8c0 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:04.210858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c37852-7da3-443d-bc87-3e494d09b8c0 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:04.217465 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82c999-ee81-4f34-b58a-0e3879bb4627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:04.219298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82c999-ee81-4f34-b58a-0e3879bb4627 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:04.219790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82c999-ee81-4f34-b58a-0e3879bb4627 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:04.220247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82c999-ee81-4f34-b58a-0e3879bb4627 admin admin] PATH_INFO: `/v3/projects/ee9262c91631413b991af2efeb169cd0/users/9ab7f2a0dcf64aa2ba8c891fc59bc8d2/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:04.259838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82c999-ee81-4f34-b58a-0e3879bb4627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9262c91631413b991af2efeb169cd0, user_id=9ab7f2a0dcf64aa2ba8c891fc59bc8d2, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:04.259838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82c999-ee81-4f34-b58a-0e3879bb4627 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:04.297619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c56fbe-17af-44e2-895c-c253ea5ad879 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:04.300039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-49c56fbe-17af-44e2-895c-c253ea5ad879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:04.320210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c56fbe-17af-44e2-895c-c253ea5ad879 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:04.324960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff2a21-f90c-46b2-9ae3-412fe2190d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:04.326392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff2a21-f90c-46b2-9ae3-412fe2190d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:04.326786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff2a21-f90c-46b2-9ae3-412fe2190d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:04.327169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff2a21-f90c-46b2-9ae3-412fe2190d86 admin admin] PATH_INFO: `/v3/users/a509668af8b44c49b273d789e8a8aac3` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:04.328715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff2a21-f90c-46b2-9ae3-412fe2190d86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a509668af8b44c49b273d789e8a8aac3)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:04.331999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff2a21-f90c-46b2-9ae3-412fe2190d86 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:04.429748 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1193/2379] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:09:04 2022] PUT /identity/v3/projects/ee9262c91631413b991af2efeb169cd0/users/9ab7f2a0dcf64aa2ba8c891fc59bc8d2/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:04.437668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f6642-53e2-41be-b022-6f0e31e124a4 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:04.437952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f6642-53e2-41be-b022-6f0e31e124a4 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:04.438227 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f6642-53e2-41be-b022-6f0e31e124a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:04.461060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-247f6642-53e2-41be-b022-6f0e31e124a4 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:04.462600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-247f6642-53e2-41be-b022-6f0e31e124a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:04.483971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:04.553493 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1194/2380] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:09:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:04.567535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f6642-53e2-41be-b022-6f0e31e124a4 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:04.571819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-247f6642-53e2-41be-b022-6f0e31e124a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:04.652954 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f6642-53e2-41be-b022-6f0e31e124a4 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:04.657224 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2860196-e702-4ce5-a019-a18bac2c7375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:04.658368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2860196-e702-4ce5-a019-a18bac2c7375 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:04.658845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2860196-e702-4ce5-a019-a18bac2c7375 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:04.659117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2860196-e702-4ce5-a019-a18bac2c7375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:04.669997 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2860196-e702-4ce5-a019-a18bac2c7375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:04.671758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2860196-e702-4ce5-a019-a18bac2c7375 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:04.692128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1195/2381] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:09:04.876057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-83ff2a21-f90c-46b2-9ae3-412fe2190d86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a509668af8b44c49b273d789e8a8aac3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:04.877148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-83ff2a21-f90c-46b2-9ae3-412fe2190d86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a509668af8b44c49b273d789e8a8aac3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:04.878092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-83ff2a21-f90c-46b2-9ae3-412fe2190d86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a509668af8b44c49b273d789e8a8aac3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:04.885757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-83ff2a21-f90c-46b2-9ae3-412fe2190d86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a509668af8b44c49b273d789e8a8aac3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:04.905372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-83ff2a21-f90c-46b2-9ae3-412fe2190d86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a509668af8b44c49b273d789e8a8aac3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:04.996942 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:05.004998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1187/2382] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:09:04 2022] DELETE /identity/v3/users/a509668af8b44c49b273d789e8a8aac3 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:05.009614 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995aad0-4f57-4685-beb4-7f8b61c99023 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:05.009816 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995aad0-4f57-4685-beb4-7f8b61c99023 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:05.010148 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995aad0-4f57-4685-beb4-7f8b61c99023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:05.034686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-3995aad0-4f57-4685-beb4-7f8b61c99023 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:05.036922 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3995aad0-4f57-4685-beb4-7f8b61c99023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:05.141127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1196/2383] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:05.160582 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff2a21-f90c-46b2-9ae3-412fe2190d86 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:05.162311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-83ff2a21-f90c-46b2-9ae3-412fe2190d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:05.245196 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff2a21-f90c-46b2-9ae3-412fe2190d86 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:05.248923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483926d-1e75-4180-9872-e41d824ebb93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:05.249591 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483926d-1e75-4180-9872-e41d824ebb93 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:05.250720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483926d-1e75-4180-9872-e41d824ebb93 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:05.250720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483926d-1e75-4180-9872-e41d824ebb93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:05.267747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483926d-1e75-4180-9872-e41d824ebb93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:05.267747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483926d-1e75-4180-9872-e41d824ebb93 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:05.282052 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1188/2384] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:09:06.012940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995aad0-4f57-4685-beb4-7f8b61c99023 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:06.014602 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3995aad0-4f57-4685-beb4-7f8b61c99023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:06.028047 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995aad0-4f57-4685-beb4-7f8b61c99023 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:06.034598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:06.035437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:06.035760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:06.049928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] PATH_INFO: `/v3/projects/d8b6c75aabb24313895a8a823917fc91` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:06.049928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b6c75aabb24313895a8a823917fc91)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:06.049928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:06.215357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b6c75aabb24313895a8a823917fc91', 'request_id': 'req-8efa9e65-309b-4283-83e4-065b85f2a0b0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:06.222308 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b6c75aabb24313895a8a823917fc91', 'request_id': 'req-8efa9e65-309b-4283-83e4-065b85f2a0b0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:06.224345 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:06.359147 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b6c75aabb24313895a8a823917fc91', 'request_id': 'req-8efa9e65-309b-4283-83e4-065b85f2a0b0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:06.371656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] The token cache is being invalidate because project d8b6c75aabb24313895a8a823917fc91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:09:06.371942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:06.372585 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:06.374177 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1197/2385] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:09:06 2022] DELETE /identity/v3/projects/d8b6c75aabb24313895a8a823917fc91 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:12.570989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226692a0-c58f-418f-9ea8-64505972948e None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:12.571596 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226692a0-c58f-418f-9ea8-64505972948e None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:12.571596 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226692a0-c58f-418f-9ea8-64505972948e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:12.604105 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-226692a0-c58f-418f-9ea8-64505972948e None None] MFA Rules not processed for user `9ab7f2a0dcf64aa2ba8c891fc59bc8d2`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:12.605344 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-226692a0-c58f-418f-9ea8-64505972948e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:12.697539 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1189/2386] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:09:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:12.702349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:12.704035 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:12.786597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa9e65-309b-4283-83e4-065b85f2a0b0 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:12.791736 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f1ba7-b827-43f8-b5cb-09f8b8938213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:12.794149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f1ba7-b827-43f8-b5cb-09f8b8938213 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:12.794149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f1ba7-b827-43f8-b5cb-09f8b8938213 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:12.794149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f1ba7-b827-43f8-b5cb-09f8b8938213 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:12.794149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f1ba7-b827-43f8-b5cb-09f8b8938213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:12.795964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f1ba7-b827-43f8-b5cb-09f8b8938213 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:12.902628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1198/2387] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:09:12 2022] POST /identity/v3/projects => generated 383 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:12.903998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226692a0-c58f-418f-9ea8-64505972948e None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:12.915606 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226692a0-c58f-418f-9ea8-64505972948e None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:13.013198 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39429f-a265-4840-ae70-bc82ff794dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:13.013909 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39429f-a265-4840-ae70-bc82ff794dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:13.014277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39429f-a265-4840-ae70-bc82ff794dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:13.014641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39429f-a265-4840-ae70-bc82ff794dc2 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:13.015481 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39429f-a265-4840-ae70-bc82ff794dc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:13.018263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39429f-a265-4840-ae70-bc82ff794dc2 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:13.142463 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1190/2388] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:09:12 2022] POST /identity/v3/users => generated 298 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:09:13.146354 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f1ba7-b827-43f8-b5cb-09f8b8938213 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:13.156584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f1ba7-b827-43f8-b5cb-09f8b8938213 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:13.161446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22af322-e8c7-4002-b19a-521b75a7d391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:13.162050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22af322-e8c7-4002-b19a-521b75a7d391 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:13.162325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22af322-e8c7-4002-b19a-521b75a7d391 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:13.162598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22af322-e8c7-4002-b19a-521b75a7d391 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:13.163290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22af322-e8c7-4002-b19a-521b75a7d391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:13.164915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22af322-e8c7-4002-b19a-521b75a7d391 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:13.176277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1199/2389] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:13.181299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39429f-a265-4840-ae70-bc82ff794dc2 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:13.201678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39429f-a265-4840-ae70-bc82ff794dc2 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:13.207112 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed43734-5fab-4724-875d-b32cd0e413a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:13.207818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed43734-5fab-4724-875d-b32cd0e413a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:13.208103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed43734-5fab-4724-875d-b32cd0e413a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:13.208376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed43734-5fab-4724-875d-b32cd0e413a9 admin admin] PATH_INFO: `/v3/projects/b46bce8c483f4d06a4c06fe36814208f/users/c1ca5918bbf844f4a512396f532211e6/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:13.224745 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed43734-5fab-4724-875d-b32cd0e413a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b46bce8c483f4d06a4c06fe36814208f, user_id=c1ca5918bbf844f4a512396f532211e6, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:13.239010 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed43734-5fab-4724-875d-b32cd0e413a9 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:13.331541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:13.338166 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1191/2390] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:09:13 2022] PUT /identity/v3/projects/b46bce8c483f4d06a4c06fe36814208f/users/c1ca5918bbf844f4a512396f532211e6/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:13.342674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22af322-e8c7-4002-b19a-521b75a7d391 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:13.353652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22af322-e8c7-4002-b19a-521b75a7d391 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:13.428864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83395f98-e715-4e1e-a398-cfa256951902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:13.429495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395f98-e715-4e1e-a398-cfa256951902 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:13.429775 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395f98-e715-4e1e-a398-cfa256951902 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:13.430049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395f98-e715-4e1e-a398-cfa256951902 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:13.430761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83395f98-e715-4e1e-a398-cfa256951902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:13.432567 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83395f98-e715-4e1e-a398-cfa256951902 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:13.443702 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1200/2391] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:09:13.452917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed43734-5fab-4724-875d-b32cd0e413a9 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:13.466579 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed43734-5fab-4724-875d-b32cd0e413a9 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:13.472213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc8853-cb04-4133-ba96-548a9cb83c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:13.472903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8853-cb04-4133-ba96-548a9cb83c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:13.473182 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8853-cb04-4133-ba96-548a9cb83c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:13.473880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8853-cb04-4133-ba96-548a9cb83c32 admin admin] PATH_INFO: `/v3/projects/b46bce8c483f4d06a4c06fe36814208f/users/c1ca5918bbf844f4a512396f532211e6/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:13.476213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc8853-cb04-4133-ba96-548a9cb83c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b46bce8c483f4d06a4c06fe36814208f, user_id=c1ca5918bbf844f4a512396f532211e6, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:13.481537 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc8853-cb04-4133-ba96-548a9cb83c32 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:13.500509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1192/2392] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:09:13 2022] PUT /identity/v3/projects/b46bce8c483f4d06a4c06fe36814208f/users/c1ca5918bbf844f4a512396f532211e6/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:09:15.639144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ec6f8-eaa5-42ca-be69-781f50fa90e3 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:15.639144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ec6f8-eaa5-42ca-be69-781f50fa90e3 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:15.639144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ec6f8-eaa5-42ca-be69-781f50fa90e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:15.666076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-757ec6f8-eaa5-42ca-be69-781f50fa90e3 None None] MFA Rules not processed for user `9cb5ce208a2442c2b9b39ac92da6a78e`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:15.668346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-757ec6f8-eaa5-42ca-be69-781f50fa90e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:15.764975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1201/2393] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:09:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:21.037810 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02d3d5-98e8-4eda-a537-2c8270f66079 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:21.037810 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02d3d5-98e8-4eda-a537-2c8270f66079 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:21.037810 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02d3d5-98e8-4eda-a537-2c8270f66079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:21.103129 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ea02d3d5-98e8-4eda-a537-2c8270f66079 None None] MFA Rules not processed for user `c1ca5918bbf844f4a512396f532211e6`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:21.105067 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ea02d3d5-98e8-4eda-a537-2c8270f66079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:21.208277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1193/2394] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:09:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:21.222004 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ec6f8-eaa5-42ca-be69-781f50fa90e3 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:21.225982 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-757ec6f8-eaa5-42ca-be69-781f50fa90e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:21.238849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:21.323093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ec6f8-eaa5-42ca-be69-781f50fa90e3 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:21.329613 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89301ba2-da6b-4fd1-8fff-a6c6e803c43a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b153935910c142aa80792e111872318a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:21.329613 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89301ba2-da6b-4fd1-8fff-a6c6e803c43a service glance] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:21.329613 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89301ba2-da6b-4fd1-8fff-a6c6e803c43a service glance] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:21.329613 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89301ba2-da6b-4fd1-8fff-a6c6e803c43a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:21.494674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89301ba2-da6b-4fd1-8fff-a6c6e803c43a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:21.498126 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89301ba2-da6b-4fd1-8fff-a6c6e803c43a service glance] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:21.515151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1202/2395] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:09:21.916772 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376df521-9937-40b4-9c78-267462f828e1 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:21.917107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376df521-9937-40b4-9c78-267462f828e1 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:21.917356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376df521-9937-40b4-9c78-267462f828e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:21.943045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-376df521-9937-40b4-9c78-267462f828e1 None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:21.944892 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-376df521-9937-40b4-9c78-267462f828e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:22.050236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1194/2396] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 15:09:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:22.077669 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89301ba2-da6b-4fd1-8fff-a6c6e803c43a service glance] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:22.079910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-89301ba2-da6b-4fd1-8fff-a6c6e803c43a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:22.179662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89301ba2-da6b-4fd1-8fff-a6c6e803c43a service glance] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:22.185238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34e890-5a57-42e9-b321-89729923cde0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:22.185874 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e890-5a57-42e9-b321-89729923cde0 service swift] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:22.186158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e890-5a57-42e9-b321-89729923cde0 service swift] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:22.186437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e890-5a57-42e9-b321-89729923cde0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:22.197097 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34e890-5a57-42e9-b321-89729923cde0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:22.199229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34e890-5a57-42e9-b321-89729923cde0 service swift] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:22.208652 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1203/2397] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:09:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:09:27.808644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376df521-9937-40b4-9c78-267462f828e1 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:27.810941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-376df521-9937-40b4-9c78-267462f828e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:27.921403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376df521-9937-40b4-9c78-267462f828e1 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:27.931646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db20af9-205f-4581-9977-99e97ea3aec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:27.932700 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db20af9-205f-4581-9977-99e97ea3aec3 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:27.933220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db20af9-205f-4581-9977-99e97ea3aec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:27.933740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db20af9-205f-4581-9977-99e97ea3aec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:27.938594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1db20af9-205f-4581-9977-99e97ea3aec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:27.994597 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:28.063525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db20af9-205f-4581-9977-99e97ea3aec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:28.068160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db20af9-205f-4581-9977-99e97ea3aec3 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:28.091316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1195/2398] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:27 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:09:30.273346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba893342-233b-406e-bf2d-b054c3fc1c9a None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:30.279025 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba893342-233b-406e-bf2d-b054c3fc1c9a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:30.279025 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba893342-233b-406e-bf2d-b054c3fc1c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:30.296557 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-ba893342-233b-406e-bf2d-b054c3fc1c9a None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:30.298722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ba893342-233b-406e-bf2d-b054c3fc1c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:30.306810 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1204/2399] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 15:09:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:09:30.343622 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db20af9-205f-4581-9977-99e97ea3aec3 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:30.376156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db20af9-205f-4581-9977-99e97ea3aec3 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:30.410857 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d53b0-6fa0-4c0a-bad7-884dddb5bfe6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:30.411927 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d53b0-6fa0-4c0a-bad7-884dddb5bfe6 service swift] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:30.414682 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d53b0-6fa0-4c0a-bad7-884dddb5bfe6 service swift] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:30.415323 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d53b0-6fa0-4c0a-bad7-884dddb5bfe6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:30.457062 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d53b0-6fa0-4c0a-bad7-884dddb5bfe6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:30.475067 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d53b0-6fa0-4c0a-bad7-884dddb5bfe6 service swift] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:30.517791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1196/2400] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:09:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:09:33.304364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178e142-9a7e-44c0-857f-d404604d1dbb None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:33.304964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178e142-9a7e-44c0-857f-d404604d1dbb None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:33.304964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178e142-9a7e-44c0-857f-d404604d1dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:33.330968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-9178e142-9a7e-44c0-857f-d404604d1dbb None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:33.332733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9178e142-9a7e-44c0-857f-d404604d1dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:33.352235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1205/2401] 10.222.0.94 () {58 vars in 967 bytes} [Tue Jul 26 15:09:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:33.396311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d53b0-6fa0-4c0a-bad7-884dddb5bfe6 service swift] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:33.411521 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d53b0-6fa0-4c0a-bad7-884dddb5bfe6 service swift] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:33.418978 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcf74b-5f57-4451-923d-475e11f7018d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:33.420258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcf74b-5f57-4451-923d-475e11f7018d service swift] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:33.420812 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcf74b-5f57-4451-923d-475e11f7018d service swift] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:33.421463 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcf74b-5f57-4451-923d-475e11f7018d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:33.437852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcf74b-5f57-4451-923d-475e11f7018d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:33.437852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcf74b-5f57-4451-923d-475e11f7018d service swift] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:33.460199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1197/2402] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:09:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:09:34.643754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9317a0-ea60-4c65-84e7-f897e9646ea5 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:34.644328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9317a0-ea60-4c65-84e7-f897e9646ea5 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:34.644328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9317a0-ea60-4c65-84e7-f897e9646ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:34.665966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-1f9317a0-ea60-4c65-84e7-f897e9646ea5 None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:34.667585 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1f9317a0-ea60-4c65-84e7-f897e9646ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:34.688890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1206/2403] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 15:09:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:34.763625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcf74b-5f57-4451-923d-475e11f7018d service swift] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:34.773681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcf74b-5f57-4451-923d-475e11f7018d service swift] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:34.788846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c82a72-f300-4344-bc4f-e1b5fef56204 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:34.789560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c82a72-f300-4344-bc4f-e1b5fef56204 service swift] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:34.789746 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c82a72-f300-4344-bc4f-e1b5fef56204 service swift] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:34.790324 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c82a72-f300-4344-bc4f-e1b5fef56204 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:34.833851 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c82a72-f300-4344-bc4f-e1b5fef56204 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:34.845219 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c82a72-f300-4344-bc4f-e1b5fef56204 service swift] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:34.885773 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1198/2404] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:09:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:09:36.143067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efccdd-fc37-43d0-9a82-08595eba358a None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:36.143067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efccdd-fc37-43d0-9a82-08595eba358a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:36.143067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efccdd-fc37-43d0-9a82-08595eba358a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:36.167520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-64efccdd-fc37-43d0-9a82-08595eba358a None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:36.169277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-64efccdd-fc37-43d0-9a82-08595eba358a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:36.185320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1207/2405] 10.222.0.94 () {58 vars in 967 bytes} [Tue Jul 26 15:09:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:36.212140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c82a72-f300-4344-bc4f-e1b5fef56204 service swift] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:36.224829 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c82a72-f300-4344-bc4f-e1b5fef56204 service swift] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:36.253416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6f407-8c70-429c-b4f1-6f6959ebd0c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:36.254125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6f407-8c70-429c-b4f1-6f6959ebd0c3 service swift] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:36.254528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6f407-8c70-429c-b4f1-6f6959ebd0c3 service swift] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:36.254941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6f407-8c70-429c-b4f1-6f6959ebd0c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:36.286974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6f407-8c70-429c-b4f1-6f6959ebd0c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:36.288779 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6f407-8c70-429c-b4f1-6f6959ebd0c3 service swift] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:36.311135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1199/2406] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:09:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:09:39.032847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb116be7-a59e-4f58-9533-ccc3e25ef3d2 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:39.033449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb116be7-a59e-4f58-9533-ccc3e25ef3d2 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:39.033449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb116be7-a59e-4f58-9533-ccc3e25ef3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:39.054186 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-cb116be7-a59e-4f58-9533-ccc3e25ef3d2 None None] MFA Rules not processed for user `9ab7f2a0dcf64aa2ba8c891fc59bc8d2`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:39.056733 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-cb116be7-a59e-4f58-9533-ccc3e25ef3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:39.064365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1208/2407] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:09:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:40.621365 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6f407-8c70-429c-b4f1-6f6959ebd0c3 service swift] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:40.622466 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-5fa6f407-8c70-429c-b4f1-6f6959ebd0c3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:40.694858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6f407-8c70-429c-b4f1-6f6959ebd0c3 service swift] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:40.699994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513634b-0eb3-400e-b5f6-b37a3f4094fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:40.700641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513634b-0eb3-400e-b5f6-b37a3f4094fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:40.700937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513634b-0eb3-400e-b5f6-b37a3f4094fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:40.701230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513634b-0eb3-400e-b5f6-b37a3f4094fd admin admin] PATH_INFO: `/v3/users/aafb57f34153410783daed91152fa43b` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:40.702679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513634b-0eb3-400e-b5f6-b37a3f4094fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=aafb57f34153410783daed91152fa43b)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:40.705602 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513634b-0eb3-400e-b5f6-b37a3f4094fd admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:40.968342 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-c513634b-0eb3-400e-b5f6-b37a3f4094fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aafb57f34153410783daed91152fa43b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:40.969195 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-c513634b-0eb3-400e-b5f6-b37a3f4094fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aafb57f34153410783daed91152fa43b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:40.970428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-c513634b-0eb3-400e-b5f6-b37a3f4094fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aafb57f34153410783daed91152fa43b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:40.981543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-c513634b-0eb3-400e-b5f6-b37a3f4094fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aafb57f34153410783daed91152fa43b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:40.996239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-c513634b-0eb3-400e-b5f6-b37a3f4094fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aafb57f34153410783daed91152fa43b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:40.998040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:41.124408 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1200/2408] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:09:40 2022] DELETE /identity/v3/users/aafb57f34153410783daed91152fa43b => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:41.128878 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387954b-0ccf-4aa9-ae4c-6cc4e0240610 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:41.129360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387954b-0ccf-4aa9-ae4c-6cc4e0240610 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:41.129512 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387954b-0ccf-4aa9-ae4c-6cc4e0240610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:41.148404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7387954b-0ccf-4aa9-ae4c-6cc4e0240610 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:41.149617 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7387954b-0ccf-4aa9-ae4c-6cc4e0240610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:41.158880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:41.217984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1209/2409] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:41.228513 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513634b-0eb3-400e-b5f6-b37a3f4094fd admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:41.235425 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c513634b-0eb3-400e-b5f6-b37a3f4094fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:41.303921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513634b-0eb3-400e-b5f6-b37a3f4094fd admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:41.307880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ba1a0-6125-4f66-90c6-542c66d528e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:41.308509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ba1a0-6125-4f66-90c6-542c66d528e2 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:41.308790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ba1a0-6125-4f66-90c6-542c66d528e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:41.309064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ba1a0-6125-4f66-90c6-542c66d528e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:41.320581 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ba1a0-6125-4f66-90c6-542c66d528e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:41.322099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ba1a0-6125-4f66-90c6-542c66d528e2 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:41.334764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1201/2410] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:09:41.637429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387954b-0ccf-4aa9-ae4c-6cc4e0240610 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:41.639273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7387954b-0ccf-4aa9-ae4c-6cc4e0240610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:41.657401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387954b-0ccf-4aa9-ae4c-6cc4e0240610 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:41.662387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ffb322-a10e-405b-939c-e98fd7f430e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:41.663281 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffb322-a10e-405b-939c-e98fd7f430e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:41.663589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffb322-a10e-405b-939c-e98fd7f430e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:41.663864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffb322-a10e-405b-939c-e98fd7f430e1 admin admin] PATH_INFO: `/v3/projects/484282e33cd64579b59888ba08ef08c5` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:41.665127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ffb322-a10e-405b-939c-e98fd7f430e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=484282e33cd64579b59888ba08ef08c5)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:41.666977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ffb322-a10e-405b-939c-e98fd7f430e1 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:41.822967 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-44ffb322-a10e-405b-939c-e98fd7f430e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '484282e33cd64579b59888ba08ef08c5', 'request_id': 'req-44ffb322-a10e-405b-939c-e98fd7f430e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:41.828826 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-44ffb322-a10e-405b-939c-e98fd7f430e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '484282e33cd64579b59888ba08ef08c5', 'request_id': 'req-44ffb322-a10e-405b-939c-e98fd7f430e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:41.940104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-44ffb322-a10e-405b-939c-e98fd7f430e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '484282e33cd64579b59888ba08ef08c5', 'request_id': 'req-44ffb322-a10e-405b-939c-e98fd7f430e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:41.952206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-44ffb322-a10e-405b-939c-e98fd7f430e1 admin admin] The token cache is being invalidate because project 484282e33cd64579b59888ba08ef08c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:09:41.952487 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-44ffb322-a10e-405b-939c-e98fd7f430e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:41.953496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-44ffb322-a10e-405b-939c-e98fd7f430e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:09:41.955227 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1210/2411] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:09:41 2022] DELETE /identity/v3/projects/484282e33cd64579b59888ba08ef08c5 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:09:44.583632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ba1a0-6125-4f66-90c6-542c66d528e2 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:44.586244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ef7ba1a0-6125-4f66-90c6-542c66d528e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:44.602357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ba1a0-6125-4f66-90c6-542c66d528e2 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:44.696214 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a60c8e-815f-4b9d-9355-15581d97e9de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:44.697262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a60c8e-815f-4b9d-9355-15581d97e9de service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:44.697628 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a60c8e-815f-4b9d-9355-15581d97e9de service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:44.698167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a60c8e-815f-4b9d-9355-15581d97e9de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:44.700650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-45a60c8e-815f-4b9d-9355-15581d97e9de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:44.836230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a60c8e-815f-4b9d-9355-15581d97e9de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:44.838923 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a60c8e-815f-4b9d-9355-15581d97e9de service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:44.860483 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1202/2412] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:09:45.572350 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a245bbd-298f-4d05-a63d-4e6c1fbd57d5 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:45.573904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a245bbd-298f-4d05-a63d-4e6c1fbd57d5 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:45.575144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a245bbd-298f-4d05-a63d-4e6c1fbd57d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:45.601265 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-6a245bbd-298f-4d05-a63d-4e6c1fbd57d5 None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:45.603630 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6a245bbd-298f-4d05-a63d-4e6c1fbd57d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:45.716633 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1211/2413] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 15:09:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:09:45.724746 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a60c8e-815f-4b9d-9355-15581d97e9de service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:45.726424 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-45a60c8e-815f-4b9d-9355-15581d97e9de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:45.791996 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:45.824372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a60c8e-815f-4b9d-9355-15581d97e9de service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:45.831561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021a579-0821-4357-9862-161e6359f773 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:45.831917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021a579-0821-4357-9862-161e6359f773 service swift] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:45.832201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021a579-0821-4357-9862-161e6359f773 service swift] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:45.832486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021a579-0821-4357-9862-161e6359f773 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:45.842449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021a579-0821-4357-9862-161e6359f773 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:45.844912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021a579-0821-4357-9862-161e6359f773 service swift] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:45.854728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1203/2414] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:09:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:09:47.362806 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a245bbd-298f-4d05-a63d-4e6c1fbd57d5 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:47.364455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6a245bbd-298f-4d05-a63d-4e6c1fbd57d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:47.393999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a245bbd-298f-4d05-a63d-4e6c1fbd57d5 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:47.401228 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5ef95-5a65-4275-b109-31807c2bbc6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:47.402959 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5ef95-5a65-4275-b109-31807c2bbc6a service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:47.403760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5ef95-5a65-4275-b109-31807c2bbc6a service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:47.404514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5ef95-5a65-4275-b109-31807c2bbc6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:47.408836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-8ae5ef95-5a65-4275-b109-31807c2bbc6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:47.508295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5ef95-5a65-4275-b109-31807c2bbc6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:47.517678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5ef95-5a65-4275-b109-31807c2bbc6a service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:47.570108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1212/2415] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:09:53.476156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186d92c-4cf0-4cc2-b307-2f07066d9cd7 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:53.487148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186d92c-4cf0-4cc2-b307-2f07066d9cd7 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:53.487148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186d92c-4cf0-4cc2-b307-2f07066d9cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:53.501109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-c186d92c-4cf0-4cc2-b307-2f07066d9cd7 None None] MFA Rules not processed for user `d2738e8ef94a48bb9a0d6b1177598112`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:09:53.502595 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c186d92c-4cf0-4cc2-b307-2f07066d9cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:53.615577 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1204/2416] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:09:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:09:57.555122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:09:57.869328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5ef95-5a65-4275-b109-31807c2bbc6a service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:09:57.870828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-8ae5ef95-5a65-4275-b109-31807c2bbc6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:57.884830 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5ef95-5a65-4275-b109-31807c2bbc6a service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:09:57.890388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7b5f0-f5f7-4bef-94ba-095be9640401 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:09:57.891025 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7b5f0-f5f7-4bef-94ba-095be9640401 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:09:57.891300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7b5f0-f5f7-4bef-94ba-095be9640401 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:09:57.891607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7b5f0-f5f7-4bef-94ba-095be9640401 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:09:57.893466 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a4e7b5f0-f5f7-4bef-94ba-095be9640401 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:09:57.903996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7b5f0-f5f7-4bef-94ba-095be9640401 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:09:57.906172 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7b5f0-f5f7-4bef-94ba-095be9640401 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:09:57.915489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1213/2417] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:09:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:10:03.592353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=41 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:10:07.911163 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=6 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:10:08.183820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186d92c-4cf0-4cc2-b307-2f07066d9cd7 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:08.183820 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c186d92c-4cf0-4cc2-b307-2f07066d9cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:08.199968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186d92c-4cf0-4cc2-b307-2f07066d9cd7 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:08.205869 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe34c7d-8d52-4253-b87a-c59a568acb4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:08.206506 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe34c7d-8d52-4253-b87a-c59a568acb4e service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:08.206801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe34c7d-8d52-4253-b87a-c59a568acb4e service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:08.207088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe34c7d-8d52-4253-b87a-c59a568acb4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:08.208980 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-afe34c7d-8d52-4253-b87a-c59a568acb4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:08.293520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe34c7d-8d52-4253-b87a-c59a568acb4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:08.294687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe34c7d-8d52-4253-b87a-c59a568acb4e service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:08.307324 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1205/2418] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:10:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:10:18.302071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=28 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:10:23.952372 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7b5f0-f5f7-4bef-94ba-095be9640401 service nova] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:23.962074 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7b5f0-f5f7-4bef-94ba-095be9640401 service nova] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:23.965877 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba7777-8953-4b77-8dc5-204a52bbe241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:23.966491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba7777-8953-4b77-8dc5-204a52bbe241 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:23.966769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba7777-8953-4b77-8dc5-204a52bbe241 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:23.967039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba7777-8953-4b77-8dc5-204a52bbe241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:23.968960 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-62ba7777-8953-4b77-8dc5-204a52bbe241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:24.038383 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba7777-8953-4b77-8dc5-204a52bbe241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:24.039847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba7777-8953-4b77-8dc5-204a52bbe241 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:24.051152 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1214/2419] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:10:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:10:26.046043 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe34c7d-8d52-4253-b87a-c59a568acb4e service nova] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:26.048911 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-afe34c7d-8d52-4253-b87a-c59a568acb4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:26.061938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe34c7d-8d52-4253-b87a-c59a568acb4e service nova] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:26.066444 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b57212-a42b-413e-8bdf-c9e34d79a2bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:26.067217 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b57212-a42b-413e-8bdf-c9e34d79a2bc service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:26.067596 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b57212-a42b-413e-8bdf-c9e34d79a2bc service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:26.067960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b57212-a42b-413e-8bdf-c9e34d79a2bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:26.070010 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c6b57212-a42b-413e-8bdf-c9e34d79a2bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:26.081589 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b57212-a42b-413e-8bdf-c9e34d79a2bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:26.083023 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b57212-a42b-413e-8bdf-c9e34d79a2bc service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:26.092007 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1206/2420] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:10:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:10:30.010527 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bcfb3-989f-4c1c-be23-a936cd3f24e3 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:30.011104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bcfb3-989f-4c1c-be23-a936cd3f24e3 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:30.011104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bcfb3-989f-4c1c-be23-a936cd3f24e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:30.045218 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b19bcfb3-989f-4c1c-be23-a936cd3f24e3 None None] MFA Rules not processed for user `61b3efca1989483d960b9f92d655d39e`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:10:30.045522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b19bcfb3-989f-4c1c-be23-a936cd3f24e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:30.072012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1215/2421] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:10:30.249047 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b57212-a42b-413e-8bdf-c9e34d79a2bc service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:30.284295 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b57212-a42b-413e-8bdf-c9e34d79a2bc service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:30.295216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dd5c4-9758-4919-ae3e-f5f288aa7b58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:30.295216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dd5c4-9758-4919-ae3e-f5f288aa7b58 service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:30.295216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dd5c4-9758-4919-ae3e-f5f288aa7b58 service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:30.295216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dd5c4-9758-4919-ae3e-f5f288aa7b58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:30.295919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-337dd5c4-9758-4919-ae3e-f5f288aa7b58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:30.311730 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337dd5c4-9758-4919-ae3e-f5f288aa7b58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:30.314762 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337dd5c4-9758-4919-ae3e-f5f288aa7b58 service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:30.328559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1207/2422] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:10:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:10:31.621046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bcfb3-989f-4c1c-be23-a936cd3f24e3 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:31.644966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bcfb3-989f-4c1c-be23-a936cd3f24e3 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:31.650637 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df4cd3-9edd-4260-873d-0b4bd5a9c005 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:31.651343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df4cd3-9edd-4260-873d-0b4bd5a9c005 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:31.651682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df4cd3-9edd-4260-873d-0b4bd5a9c005 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:31.651961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df4cd3-9edd-4260-873d-0b4bd5a9c005 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:31.653818 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-22df4cd3-9edd-4260-873d-0b4bd5a9c005 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:31.679245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df4cd3-9edd-4260-873d-0b4bd5a9c005 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:31.680886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df4cd3-9edd-4260-873d-0b4bd5a9c005 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:31.691841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1216/2423] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:10:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:10:40.321990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=17 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:10:41.687120 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=58 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:10:57.826715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dd5c4-9758-4919-ae3e-f5f288aa7b58 service nova] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:10:57.828759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-337dd5c4-9758-4919-ae3e-f5f288aa7b58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:57.925539 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dd5c4-9758-4919-ae3e-f5f288aa7b58 service nova] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:10:57.929319 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11475b6-1178-4db1-9ec4-dd33120b163c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:10:57.929936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11475b6-1178-4db1-9ec4-dd33120b163c service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:10:57.930216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11475b6-1178-4db1-9ec4-dd33120b163c service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:10:57.930531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11475b6-1178-4db1-9ec4-dd33120b163c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:10:57.932313 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d11475b6-1178-4db1-9ec4-dd33120b163c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:10:57.994737 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11475b6-1178-4db1-9ec4-dd33120b163c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:10:57.996218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11475b6-1178-4db1-9ec4-dd33120b163c service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:10:58.005758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1208/2424] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:10:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:11:03.278465 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df4cd3-9edd-4260-873d-0b4bd5a9c005 service nova] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:11:03.280015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-22df4cd3-9edd-4260-873d-0b4bd5a9c005 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:03.382754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df4cd3-9edd-4260-873d-0b4bd5a9c005 service nova] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:11:03.387567 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5791b-f1da-484f-85f0-8d4c8698997e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:11:03.388207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5791b-f1da-484f-85f0-8d4c8698997e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:03.388485 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5791b-f1da-484f-85f0-8d4c8698997e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:03.388865 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5791b-f1da-484f-85f0-8d4c8698997e admin admin] PATH_INFO: `/v3/users/9cb5ce208a2442c2b9b39ac92da6a78e` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:03.390058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5791b-f1da-484f-85f0-8d4c8698997e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cb5ce208a2442c2b9b39ac92da6a78e)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:11:03.392001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5791b-f1da-484f-85f0-8d4c8698997e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:11:03.607231 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-95f5791b-f1da-484f-85f0-8d4c8698997e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cb5ce208a2442c2b9b39ac92da6a78e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:03.615475 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-95f5791b-f1da-484f-85f0-8d4c8698997e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cb5ce208a2442c2b9b39ac92da6a78e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:03.710601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-95f5791b-f1da-484f-85f0-8d4c8698997e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cb5ce208a2442c2b9b39ac92da6a78e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:03.724247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-95f5791b-f1da-484f-85f0-8d4c8698997e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cb5ce208a2442c2b9b39ac92da6a78e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:03.724901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-95f5791b-f1da-484f-85f0-8d4c8698997e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cb5ce208a2442c2b9b39ac92da6a78e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:03.730805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1217/2425] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:11:03 2022] DELETE /identity/v3/users/9cb5ce208a2442c2b9b39ac92da6a78e => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:11:04.020940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11475b6-1178-4db1-9ec4-dd33120b163c service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:11:04.035629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11475b6-1178-4db1-9ec4-dd33120b163c service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:11:04.100053 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:11:04.101297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:04.101583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:04.101858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] PATH_INFO: `/v3/projects/87391d9e4bdc48209437f1d786f376c1` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:04.103054 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87391d9e4bdc48209437f1d786f376c1)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:11:04.105085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:11:04.131658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:11:04.244549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87391d9e4bdc48209437f1d786f376c1', 'request_id': 'req-d21ab9a1-d2e8-40de-9f73-69932d2bc292'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:04.253264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87391d9e4bdc48209437f1d786f376c1', 'request_id': 'req-d21ab9a1-d2e8-40de-9f73-69932d2bc292'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:04.259708 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87391d9e4bdc48209437f1d786f376c1', 'request_id': 'req-d21ab9a1-d2e8-40de-9f73-69932d2bc292'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:04.355965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] The token cache is being invalidate because project 87391d9e4bdc48209437f1d786f376c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:11:04.356383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:04.362755 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:04.362755 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1209/2426] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:11:04 2022] DELETE /identity/v3/projects/87391d9e4bdc48209437f1d786f376c1 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:11:08.306814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da54bb-7baa-4d50-b7d4-5ce27b8d7aff None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:08.307394 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da54bb-7baa-4d50-b7d4-5ce27b8d7aff None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:08.307394 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da54bb-7baa-4d50-b7d4-5ce27b8d7aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:08.329200 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-a6da54bb-7baa-4d50-b7d4-5ce27b8d7aff None None] MFA Rules not processed for user `a1bc5bf55e9d452b88757ea4080165e0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:11:08.330417 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a6da54bb-7baa-4d50-b7d4-5ce27b8d7aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:08.377254 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:11:08.387661 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1218/2427] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:11:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:11:08.706371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:11:08.708022 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:08.790915 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ab9a1-d2e8-40de-9f73-69932d2bc292 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:11:08.795333 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daaa8de-c766-43e3-bf7e-c854065f34db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:11:08.795981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaa8de-c766-43e3-bf7e-c854065f34db service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:08.796762 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaa8de-c766-43e3-bf7e-c854065f34db service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:08.797036 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daaa8de-c766-43e3-bf7e-c854065f34db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:08.798901 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0daaa8de-c766-43e3-bf7e-c854065f34db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:08.810762 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daaa8de-c766-43e3-bf7e-c854065f34db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:11:08.813309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daaa8de-c766-43e3-bf7e-c854065f34db service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:11:08.822668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1210/2428] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:11:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:11:09.254684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da54bb-7baa-4d50-b7d4-5ce27b8d7aff None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:11:09.256266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a6da54bb-7baa-4d50-b7d4-5ce27b8d7aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:09.337059 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da54bb-7baa-4d50-b7d4-5ce27b8d7aff None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:11:09.342220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98609f7-11df-4b90-a8f3-297a6be5b888 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a0728e2f6249bdb45d6f44fb15aa4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:11:09.342859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98609f7-11df-4b90-a8f3-297a6be5b888 service placement] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:09.344030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98609f7-11df-4b90-a8f3-297a6be5b888 service placement] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:09.344379 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98609f7-11df-4b90-a8f3-297a6be5b888 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:09.346184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e98609f7-11df-4b90-a8f3-297a6be5b888 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:09.358221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98609f7-11df-4b90-a8f3-297a6be5b888 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:11:09.360695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98609f7-11df-4b90-a8f3-297a6be5b888 service placement] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:11:09.373137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1219/2429] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:11:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:11:18.818410 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:DELETE=7,keystone:SELECT=50,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:11:19.367309 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=34 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:11:20.548506 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daaa8de-c766-43e3-bf7e-c854065f34db service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:11:20.550467 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0daaa8de-c766-43e3-bf7e-c854065f34db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:20.628735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daaa8de-c766-43e3-bf7e-c854065f34db service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:11:20.634445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930b80a-2bd4-43d9-ad46-f1ebabbf4b6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:11:20.635096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930b80a-2bd4-43d9-ad46-f1ebabbf4b6a service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:20.635380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930b80a-2bd4-43d9-ad46-f1ebabbf4b6a service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:20.635692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930b80a-2bd4-43d9-ad46-f1ebabbf4b6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:20.637548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e930b80a-2bd4-43d9-ad46-f1ebabbf4b6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:20.650369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930b80a-2bd4-43d9-ad46-f1ebabbf4b6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:11:20.652946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930b80a-2bd4-43d9-ad46-f1ebabbf4b6a service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:11:20.664607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1211/2430] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:11:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:11:30.659311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=34 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:11:40.490501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33690c82-e489-456e-9927-bf30747741a0 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:40.491424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33690c82-e489-456e-9927-bf30747741a0 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:40.491738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33690c82-e489-456e-9927-bf30747741a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:40.512040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-33690c82-e489-456e-9927-bf30747741a0 None None] MFA Rules not processed for user `d2738e8ef94a48bb9a0d6b1177598112`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:11:40.513960 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-33690c82-e489-456e-9927-bf30747741a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:40.584786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1220/2431] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:11:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:11:40.930125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930b80a-2bd4-43d9-ad46-f1ebabbf4b6a service nova] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:11:40.931901 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e930b80a-2bd4-43d9-ad46-f1ebabbf4b6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:40.945417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930b80a-2bd4-43d9-ad46-f1ebabbf4b6a service nova] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:11:40.959015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b8827-ae76-405d-a0e1-47c7c1fa27b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:11:40.959015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b8827-ae76-405d-a0e1-47c7c1fa27b7 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:40.959015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b8827-ae76-405d-a0e1-47c7c1fa27b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:40.959015 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b8827-ae76-405d-a0e1-47c7c1fa27b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:40.959494 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9f7b8827-ae76-405d-a0e1-47c7c1fa27b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:40.970734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b8827-ae76-405d-a0e1-47c7c1fa27b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:11:40.972238 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b8827-ae76-405d-a0e1-47c7c1fa27b7 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:11:40.983791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1212/2432] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:11:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:11:50.576793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=25 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:11:50.977118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=6 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:11:53.624672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33690c82-e489-456e-9927-bf30747741a0 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:11:53.636298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33690c82-e489-456e-9927-bf30747741a0 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:11:53.640216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15701b1c-1ebf-4730-aa98-d37a10c41366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:11:53.640844 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15701b1c-1ebf-4730-aa98-d37a10c41366 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:53.641121 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15701b1c-1ebf-4730-aa98-d37a10c41366 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:53.641388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15701b1c-1ebf-4730-aa98-d37a10c41366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:53.643212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-15701b1c-1ebf-4730-aa98-d37a10c41366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:53.716842 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15701b1c-1ebf-4730-aa98-d37a10c41366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:11:53.719148 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15701b1c-1ebf-4730-aa98-d37a10c41366 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:11:53.729681 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1221/2433] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:11:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:11:58.307436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b8827-ae76-405d-a0e1-47c7c1fa27b7 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:11:58.310430 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9f7b8827-ae76-405d-a0e1-47c7c1fa27b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:58.326499 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b8827-ae76-405d-a0e1-47c7c1fa27b7 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:11:58.334797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de46c9-647f-4f7f-844f-b9ac334f0286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:11:58.334797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de46c9-647f-4f7f-844f-b9ac334f0286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:58.334797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de46c9-647f-4f7f-844f-b9ac334f0286 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:58.334797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de46c9-647f-4f7f-844f-b9ac334f0286 admin admin] PATH_INFO: `/v3/users/61b3efca1989483d960b9f92d655d39e` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:58.335516 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de46c9-647f-4f7f-844f-b9ac334f0286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61b3efca1989483d960b9f92d655d39e)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:11:58.338193 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de46c9-647f-4f7f-844f-b9ac334f0286 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:11:58.581886 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b0de46c9-647f-4f7f-844f-b9ac334f0286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61b3efca1989483d960b9f92d655d39e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:58.582663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b0de46c9-647f-4f7f-844f-b9ac334f0286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61b3efca1989483d960b9f92d655d39e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:58.583311 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b0de46c9-647f-4f7f-844f-b9ac334f0286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61b3efca1989483d960b9f92d655d39e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:58.591886 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b0de46c9-647f-4f7f-844f-b9ac334f0286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61b3efca1989483d960b9f92d655d39e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:58.606048 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b0de46c9-647f-4f7f-844f-b9ac334f0286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61b3efca1989483d960b9f92d655d39e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:58.721230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1213/2434] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:11:58 2022] DELETE /identity/v3/users/61b3efca1989483d960b9f92d655d39e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:11:59.046633 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15701b1c-1ebf-4730-aa98-d37a10c41366 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:11:59.049755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15701b1c-1ebf-4730-aa98-d37a10c41366 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:11:59.136728 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d9da7-8f04-41fd-a48c-d7a01f9cde43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:11:59.137378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d9da7-8f04-41fd-a48c-d7a01f9cde43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:59.138078 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d9da7-8f04-41fd-a48c-d7a01f9cde43 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:59.138505 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d9da7-8f04-41fd-a48c-d7a01f9cde43 admin admin] PATH_INFO: `/v3/projects/dbe01616efc64d2e901f45662e7c3518` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:59.139757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d9da7-8f04-41fd-a48c-d7a01f9cde43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbe01616efc64d2e901f45662e7c3518)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:11:59.141671 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d9da7-8f04-41fd-a48c-d7a01f9cde43 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:11:59.276014 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-154d9da7-8f04-41fd-a48c-d7a01f9cde43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbe01616efc64d2e901f45662e7c3518', 'request_id': 'req-154d9da7-8f04-41fd-a48c-d7a01f9cde43'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:59.283197 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-154d9da7-8f04-41fd-a48c-d7a01f9cde43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbe01616efc64d2e901f45662e7c3518', 'request_id': 'req-154d9da7-8f04-41fd-a48c-d7a01f9cde43'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:59.349779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-154d9da7-8f04-41fd-a48c-d7a01f9cde43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbe01616efc64d2e901f45662e7c3518', 'request_id': 'req-154d9da7-8f04-41fd-a48c-d7a01f9cde43'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:59.366775 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-154d9da7-8f04-41fd-a48c-d7a01f9cde43 admin admin] The token cache is being invalidate because project dbe01616efc64d2e901f45662e7c3518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:11:59.367067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-154d9da7-8f04-41fd-a48c-d7a01f9cde43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:59.367782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-154d9da7-8f04-41fd-a48c-d7a01f9cde43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:11:59.369397 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1222/2435] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:11:59 2022] DELETE /identity/v3/projects/dbe01616efc64d2e901f45662e7c3518 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:11:59.375819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b5125-4134-41a1-81af-7a9feb883736 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:59.376528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b5125-4134-41a1-81af-7a9feb883736 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:59.376677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b5125-4134-41a1-81af-7a9feb883736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:59.441114 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-a08b5125-4134-41a1-81af-7a9feb883736 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:11:59.442344 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a08b5125-4134-41a1-81af-7a9feb883736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:59.510886 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1214/2436] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:11:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:11:59.519620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa54f8-5ca2-4c70-9836-bb0f83cd8e6a None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:59.519909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa54f8-5ca2-4c70-9836-bb0f83cd8e6a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:59.520191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa54f8-5ca2-4c70-9836-bb0f83cd8e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:59.544524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-adaa54f8-5ca2-4c70-9836-bb0f83cd8e6a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:11:59.545888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-adaa54f8-5ca2-4c70-9836-bb0f83cd8e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:11:59.564449 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1223/2437] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:11:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:11:59.571358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b5125-4134-41a1-81af-7a9feb883736 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:11:59.582879 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b5125-4134-41a1-81af-7a9feb883736 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:11:59.588751 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa212fb-77cb-4df6-997c-ad2a6779fac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:11:59.589431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa212fb-77cb-4df6-997c-ad2a6779fac2 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:59.589708 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa212fb-77cb-4df6-997c-ad2a6779fac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:59.590165 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa212fb-77cb-4df6-997c-ad2a6779fac2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:59.590848 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa212fb-77cb-4df6-997c-ad2a6779fac2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:11:59.591189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa212fb-77cb-4df6-997c-ad2a6779fac2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:11:59.593227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa212fb-77cb-4df6-997c-ad2a6779fac2 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:11:59.608121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1215/2438] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:11:59.612724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa54f8-5ca2-4c70-9836-bb0f83cd8e6a None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:11:59.624410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa54f8-5ca2-4c70-9836-bb0f83cd8e6a None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:11:59.629767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a4492-6e2d-4a31-b6c9-df541834f6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:11:59.630606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a4492-6e2d-4a31-b6c9-df541834f6ff admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:59.630998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a4492-6e2d-4a31-b6c9-df541834f6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:59.631335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a4492-6e2d-4a31-b6c9-df541834f6ff admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:59.632139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a4492-6e2d-4a31-b6c9-df541834f6ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:11:59.634442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a4492-6e2d-4a31-b6c9-df541834f6ff admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:11:59.659132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:11:59.765992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1224/2439] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:11:59 2022] POST /identity/v3/projects => generated 389 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:11:59.767053 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa212fb-77cb-4df6-997c-ad2a6779fac2 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:11:59.777263 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa212fb-77cb-4df6-997c-ad2a6779fac2 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:11:59.812353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:11:59.847438 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e4094-f3f3-41d2-9fa0-e9b1c5f88a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:11:59.850046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e4094-f3f3-41d2-9fa0-e9b1c5f88a08 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:59.850046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e4094-f3f3-41d2-9fa0-e9b1c5f88a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:59.850046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e4094-f3f3-41d2-9fa0-e9b1c5f88a08 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:59.850046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e4094-f3f3-41d2-9fa0-e9b1c5f88a08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:11:59.851285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e4094-f3f3-41d2-9fa0-e9b1c5f88a08 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:11:59.945208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1216/2440] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:11:59 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:11:59.949346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a4492-6e2d-4a31-b6c9-df541834f6ff admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:11:59.962566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a4492-6e2d-4a31-b6c9-df541834f6ff admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:11:59.970377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d37f19-3579-4b9c-a33b-8ea92582bdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:11:59.970377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d37f19-3579-4b9c-a33b-8ea92582bdd2 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:11:59.970742 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d37f19-3579-4b9c-a33b-8ea92582bdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:11:59.971088 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d37f19-3579-4b9c-a33b-8ea92582bdd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:11:59.971842 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d37f19-3579-4b9c-a33b-8ea92582bdd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:11:59.973478 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d37f19-3579-4b9c-a33b-8ea92582bdd2 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:11:59.990849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1225/2441] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:11:59.991544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e4094-f3f3-41d2-9fa0-e9b1c5f88a08 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:00.003189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e4094-f3f3-41d2-9fa0-e9b1c5f88a08 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:00.008939 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382684eb-98a0-4191-9d50-d7337b5f4912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:00.009699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382684eb-98a0-4191-9d50-d7337b5f4912 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:00.009992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382684eb-98a0-4191-9d50-d7337b5f4912 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:00.010308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382684eb-98a0-4191-9d50-d7337b5f4912 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:00.011021 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382684eb-98a0-4191-9d50-d7337b5f4912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:00.013831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382684eb-98a0-4191-9d50-d7337b5f4912 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:00.026698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1217/2442] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:12:00.031094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d37f19-3579-4b9c-a33b-8ea92582bdd2 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:00.042278 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d37f19-3579-4b9c-a33b-8ea92582bdd2 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:00.049812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab745365-539e-4a56-9401-df7fa33b8783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:00.051451 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab745365-539e-4a56-9401-df7fa33b8783 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:00.056572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab745365-539e-4a56-9401-df7fa33b8783 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:00.056572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab745365-539e-4a56-9401-df7fa33b8783 admin admin] PATH_INFO: `/v3/projects/b57441fed5b5437581930346bf9116ef/users/b7e467910de54b7ea55cb2d728a8c9be/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:00.085595 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab745365-539e-4a56-9401-df7fa33b8783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b57441fed5b5437581930346bf9116ef, user_id=b7e467910de54b7ea55cb2d728a8c9be, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:00.091376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab745365-539e-4a56-9401-df7fa33b8783 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:00.204685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1226/2443] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:12:00 2022] PUT /identity/v3/projects/b57441fed5b5437581930346bf9116ef/users/b7e467910de54b7ea55cb2d728a8c9be/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:12:00.209807 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a3743-65a8-46fa-b7a6-9e3d8d0c2fe5 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:00.210095 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a3743-65a8-46fa-b7a6-9e3d8d0c2fe5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:00.210379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a3743-65a8-46fa-b7a6-9e3d8d0c2fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:00.233373 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-be4a3743-65a8-46fa-b7a6-9e3d8d0c2fe5 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:12:00.234630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-be4a3743-65a8-46fa-b7a6-9e3d8d0c2fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:00.332543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1218/2444] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:12:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:12:00.343093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab745365-539e-4a56-9401-df7fa33b8783 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:00.344940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ab745365-539e-4a56-9401-df7fa33b8783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:00.419686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab745365-539e-4a56-9401-df7fa33b8783 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:00.424995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6feb77-8f1c-4343-96d1-4027fc2b4c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:00.426247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6feb77-8f1c-4343-96d1-4027fc2b4c71 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:00.426815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6feb77-8f1c-4343-96d1-4027fc2b4c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:00.427347 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6feb77-8f1c-4343-96d1-4027fc2b4c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:00.440644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6feb77-8f1c-4343-96d1-4027fc2b4c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:00.443098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6feb77-8f1c-4343-96d1-4027fc2b4c71 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:00.457273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1227/2445] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:12:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:12:03.206236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88665c9b-df8d-4df5-907f-a098fdea2090 None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:03.206831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88665c9b-df8d-4df5-907f-a098fdea2090 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:03.206831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88665c9b-df8d-4df5-907f-a098fdea2090 None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:03.209898 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1219/2446] 10.222.0.94 () {54 vars in 921 bytes} [Tue Jul 26 15:12:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:12:03.217403 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb1ac8-dd73-434a-bf9a-3e2052e00233 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:03.218023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb1ac8-dd73-434a-bf9a-3e2052e00233 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:03.218023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb1ac8-dd73-434a-bf9a-3e2052e00233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:03.267985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-e3eb1ac8-dd73-434a-bf9a-3e2052e00233 None None] MFA Rules not processed for user `d39fd2708591424ca4892973e7da48ee`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:12:03.269583 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e3eb1ac8-dd73-434a-bf9a-3e2052e00233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:03.277710 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1228/2447] 10.222.0.94 () {58 vars in 1018 bytes} [Tue Jul 26 15:12:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:12:03.295012 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88665c9b-df8d-4df5-907f-a098fdea2090 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:03.317116 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88665c9b-df8d-4df5-907f-a098fdea2090 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:03.320975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e43d0a-4897-441f-9a1f-a4afc8a0a801 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:03.322431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e43d0a-4897-441f-9a1f-a4afc8a0a801 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:03.322431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e43d0a-4897-441f-9a1f-a4afc8a0a801 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:03.322431 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e43d0a-4897-441f-9a1f-a4afc8a0a801 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:03.331185 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e43d0a-4897-441f-9a1f-a4afc8a0a801 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:03.334694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e43d0a-4897-441f-9a1f-a4afc8a0a801 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:03.342686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1220/2448] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:12:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:12:03.511676 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a5913-99f7-434b-9c58-a9636c77eaae None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:03.512280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a5913-99f7-434b-9c58-a9636c77eaae None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:03.512628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a5913-99f7-434b-9c58-a9636c77eaae None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:03.514670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1229/2449] 10.222.0.94 () {54 vars in 921 bytes} [Tue Jul 26 15:12:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:12:03.520781 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826425e2-7a85-447a-b07b-c5f395753a55 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:03.521067 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826425e2-7a85-447a-b07b-c5f395753a55 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:03.521345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826425e2-7a85-447a-b07b-c5f395753a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:03.549646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-826425e2-7a85-447a-b07b-c5f395753a55 None None] MFA Rules not processed for user `6a4278fe995e432d88f4346d6f54218b`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:12:03.550940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-826425e2-7a85-447a-b07b-c5f395753a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:03.633527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1221/2450] 10.222.0.94 () {58 vars in 1018 bytes} [Tue Jul 26 15:12:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:12:03.644723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a5913-99f7-434b-9c58-a9636c77eaae None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:03.646381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-822a5913-99f7-434b-9c58-a9636c77eaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:03.662398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a5913-99f7-434b-9c58-a9636c77eaae None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:03.666454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d984a-2d0a-432a-b45e-d46c0ccd6d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:03.667285 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d984a-2d0a-432a-b45e-d46c0ccd6d5b service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:03.667480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d984a-2d0a-432a-b45e-d46c0ccd6d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:03.667764 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d984a-2d0a-432a-b45e-d46c0ccd6d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:03.679740 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d984a-2d0a-432a-b45e-d46c0ccd6d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:03.681302 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d984a-2d0a-432a-b45e-d46c0ccd6d5b service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:03.693519 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1230/2451] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:12:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:12:06.719044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164248b5-b6ff-4019-8538-4c7f610d19ae None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:06.719639 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164248b5-b6ff-4019-8538-4c7f610d19ae None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:06.719741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164248b5-b6ff-4019-8538-4c7f610d19ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:06.753358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-164248b5-b6ff-4019-8538-4c7f610d19ae None None] MFA Rules not processed for user `b7e467910de54b7ea55cb2d728a8c9be`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:12:06.754790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-164248b5-b6ff-4019-8538-4c7f610d19ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:06.775183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:12:06.862064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1222/2452] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:12:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:12:06.880763 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d984a-2d0a-432a-b45e-d46c0ccd6d5b service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:06.882294 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b06d984a-2d0a-432a-b45e-d46c0ccd6d5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:06.899054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d984a-2d0a-432a-b45e-d46c0ccd6d5b service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:06.921938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52640f73-d14b-40a7-934f-2c4be6d19764 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:06.923044 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52640f73-d14b-40a7-934f-2c4be6d19764 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:06.923548 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52640f73-d14b-40a7-934f-2c4be6d19764 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:06.924007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52640f73-d14b-40a7-934f-2c4be6d19764 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:06.938466 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52640f73-d14b-40a7-934f-2c4be6d19764 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:06.940170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52640f73-d14b-40a7-934f-2c4be6d19764 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:06.950326 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1231/2453] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:12:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:12:09.773983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8d023-8119-4cb0-be59-f164545ad2e9 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:09.774567 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8d023-8119-4cb0-be59-f164545ad2e9 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:09.774799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8d023-8119-4cb0-be59-f164545ad2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:09.795880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-05b8d023-8119-4cb0-be59-f164545ad2e9 None None] MFA Rules not processed for user `b7e467910de54b7ea55cb2d728a8c9be`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:12:09.797375 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-05b8d023-8119-4cb0-be59-f164545ad2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:09.811436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1223/2454] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:12:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:12:16.946463 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=65,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:12:19.790044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=23 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:12:38.270944 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa90d5-e257-454a-b9f3-6978305eae3b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:38.271516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa90d5-e257-454a-b9f3-6978305eae3b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:38.271516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa90d5-e257-454a-b9f3-6978305eae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:38.293260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-47fa90d5-e257-454a-b9f3-6978305eae3b None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:12:38.293965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-47fa90d5-e257-454a-b9f3-6978305eae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:38.376609 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1232/2455] 10.222.0.94 () {58 vars in 967 bytes} [Tue Jul 26 15:12:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:12:38.423009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8d023-8119-4cb0-be59-f164545ad2e9 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:38.424534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-05b8d023-8119-4cb0-be59-f164545ad2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:38.509669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8d023-8119-4cb0-be59-f164545ad2e9 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:38.515286 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8397b-a1c6-490b-8411-70b9d1e42ab3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:38.515963 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8397b-a1c6-490b-8411-70b9d1e42ab3 service swift] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:38.516252 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8397b-a1c6-490b-8411-70b9d1e42ab3 service swift] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:38.516534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8397b-a1c6-490b-8411-70b9d1e42ab3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:38.526203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec8397b-a1c6-490b-8411-70b9d1e42ab3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:38.527993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec8397b-a1c6-490b-8411-70b9d1e42ab3 service swift] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:38.536863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1224/2456] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:12:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:12:47.602708 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa90d5-e257-454a-b9f3-6978305eae3b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:47.605726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-47fa90d5-e257-454a-b9f3-6978305eae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:47.625179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa90d5-e257-454a-b9f3-6978305eae3b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:47.635298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fd486-1924-45d1-a835-92fc404155be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:47.636531 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fd486-1924-45d1-a835-92fc404155be admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:47.636937 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fd486-1924-45d1-a835-92fc404155be admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:47.637212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fd486-1924-45d1-a835-92fc404155be admin admin] PATH_INFO: `/v3/users/9ab7f2a0dcf64aa2ba8c891fc59bc8d2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:47.638452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fd486-1924-45d1-a835-92fc404155be admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab7f2a0dcf64aa2ba8c891fc59bc8d2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:47.640518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fd486-1924-45d1-a835-92fc404155be admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:47.964209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-672fd486-1924-45d1-a835-92fc404155be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab7f2a0dcf64aa2ba8c891fc59bc8d2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:47.992515 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-672fd486-1924-45d1-a835-92fc404155be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab7f2a0dcf64aa2ba8c891fc59bc8d2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:48.073600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-672fd486-1924-45d1-a835-92fc404155be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab7f2a0dcf64aa2ba8c891fc59bc8d2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:48.086675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-672fd486-1924-45d1-a835-92fc404155be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab7f2a0dcf64aa2ba8c891fc59bc8d2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:48.087454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-672fd486-1924-45d1-a835-92fc404155be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab7f2a0dcf64aa2ba8c891fc59bc8d2'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:48.098614 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1233/2457] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:12:47 2022] DELETE /identity/v3/users/9ab7f2a0dcf64aa2ba8c891fc59bc8d2 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:12:48.363046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8397b-a1c6-490b-8411-70b9d1e42ab3 service swift] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:48.392919 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8397b-a1c6-490b-8411-70b9d1e42ab3 service swift] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:48.519185 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:48.522099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:48.522099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:48.522099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] PATH_INFO: `/v3/projects/ee9262c91631413b991af2efeb169cd0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:48.523283 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee9262c91631413b991af2efeb169cd0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:48.526334 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:48.684312 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee9262c91631413b991af2efeb169cd0', 'request_id': 'req-8d03313c-08b3-4753-8dc5-0a5433f66354'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:48.693062 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee9262c91631413b991af2efeb169cd0', 'request_id': 'req-8d03313c-08b3-4753-8dc5-0a5433f66354'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:48.703281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee9262c91631413b991af2efeb169cd0', 'request_id': 'req-8d03313c-08b3-4753-8dc5-0a5433f66354'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:48.853794 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] The token cache is being invalidate because project ee9262c91631413b991af2efeb169cd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:12:48.854189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:48.855205 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:48.856887 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1225/2458] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:12:48 2022] DELETE /identity/v3/projects/ee9262c91631413b991af2efeb169cd0 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:12:48.861220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fd486-1924-45d1-a835-92fc404155be admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:48.863500 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-672fd486-1924-45d1-a835-92fc404155be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:48.880001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fd486-1924-45d1-a835-92fc404155be admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:48.944043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093af99-fb75-4e9c-88d8-d6145af7f682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:48.944711 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093af99-fb75-4e9c-88d8-d6145af7f682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:48.944998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093af99-fb75-4e9c-88d8-d6145af7f682 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:48.945276 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093af99-fb75-4e9c-88d8-d6145af7f682 admin admin] PATH_INFO: `/v3/users/c1ca5918bbf844f4a512396f532211e6` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:48.946551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093af99-fb75-4e9c-88d8-d6145af7f682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ca5918bbf844f4a512396f532211e6)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:48.949156 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093af99-fb75-4e9c-88d8-d6145af7f682 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:48.963230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:12:49.018118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:49.019651 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:49.050292 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:12:49.138675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5093af99-fb75-4e9c-88d8-d6145af7f682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ca5918bbf844f4a512396f532211e6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:49.148347 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5093af99-fb75-4e9c-88d8-d6145af7f682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ca5918bbf844f4a512396f532211e6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:49.162272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03313c-08b3-4753-8dc5-0a5433f66354 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:49.169475 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2118f87-3c42-41c5-8806-171a4c42fb9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a0728e2f6249bdb45d6f44fb15aa4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:49.170321 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2118f87-3c42-41c5-8806-171a4c42fb9f service placement] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:49.170630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2118f87-3c42-41c5-8806-171a4c42fb9f service placement] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:49.170910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2118f87-3c42-41c5-8806-171a4c42fb9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:49.172851 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d2118f87-3c42-41c5-8806-171a4c42fb9f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:49.186742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2118f87-3c42-41c5-8806-171a4c42fb9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:49.188984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2118f87-3c42-41c5-8806-171a4c42fb9f service placement] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:49.205785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1226/2459] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:12:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:12:49.222033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5093af99-fb75-4e9c-88d8-d6145af7f682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ca5918bbf844f4a512396f532211e6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:49.238861 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5093af99-fb75-4e9c-88d8-d6145af7f682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ca5918bbf844f4a512396f532211e6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:49.240618 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5093af99-fb75-4e9c-88d8-d6145af7f682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ca5918bbf844f4a512396f532211e6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:49.265698 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1234/2460] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:12:48 2022] DELETE /identity/v3/users/c1ca5918bbf844f4a512396f532211e6 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:12:49.560669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2118f87-3c42-41c5-8806-171a4c42fb9f service placement] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:49.562163 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d2118f87-3c42-41c5-8806-171a4c42fb9f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:49.579354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2118f87-3c42-41c5-8806-171a4c42fb9f service placement] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:49.657413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:49.658139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:49.658427 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:49.658706 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] PATH_INFO: `/v3/projects/b46bce8c483f4d06a4c06fe36814208f` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:49.659971 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] RBAC: Authorizing `identity:delete_project(project_id=b46bce8c483f4d06a4c06fe36814208f)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:49.661894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:49.824722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b46bce8c483f4d06a4c06fe36814208f', 'request_id': 'req-4caf8034-3a6b-4277-9956-d4319c3169df'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:49.831560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b46bce8c483f4d06a4c06fe36814208f', 'request_id': 'req-4caf8034-3a6b-4277-9956-d4319c3169df'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:49.837257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b46bce8c483f4d06a4c06fe36814208f', 'request_id': 'req-4caf8034-3a6b-4277-9956-d4319c3169df'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:49.893966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] The token cache is being invalidate because project b46bce8c483f4d06a4c06fe36814208f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:12:49.894492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:49.898531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:12:49.900177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1227/2461] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:12:49 2022] DELETE /identity/v3/projects/b46bce8c483f4d06a4c06fe36814208f => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:12:56.142944 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093af99-fb75-4e9c-88d8-d6145af7f682 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:56.144685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5093af99-fb75-4e9c-88d8-d6145af7f682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:56.204763 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093af99-fb75-4e9c-88d8-d6145af7f682 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:12:56.256783 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c71ad-6623-49a5-9e54-2a5f53bb5e20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:12:56.257399 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c71ad-6623-49a5-9e54-2a5f53bb5e20 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:12:56.257682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c71ad-6623-49a5-9e54-2a5f53bb5e20 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:12:56.257958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c71ad-6623-49a5-9e54-2a5f53bb5e20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:12:56.259779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f03c71ad-6623-49a5-9e54-2a5f53bb5e20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:12:56.324390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:12:56.326864 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c71ad-6623-49a5-9e54-2a5f53bb5e20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:12:56.328324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c71ad-6623-49a5-9e54-2a5f53bb5e20 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:12:56.338209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1235/2462] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:12:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:12:59.887270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:12:59.895863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:12:59.897569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:00.013149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf8034-3a6b-4277-9956-d4319c3169df admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:00.022664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75218a62-40c0-45a1-b27a-792a8d3db312 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a0728e2f6249bdb45d6f44fb15aa4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:00.022664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75218a62-40c0-45a1-b27a-792a8d3db312 service placement] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:00.022664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75218a62-40c0-45a1-b27a-792a8d3db312 service placement] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:00.022664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75218a62-40c0-45a1-b27a-792a8d3db312 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:00.023250 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-75218a62-40c0-45a1-b27a-792a8d3db312 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:00.037606 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75218a62-40c0-45a1-b27a-792a8d3db312 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:00.051803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75218a62-40c0-45a1-b27a-792a8d3db312 service placement] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:00.053221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c71ad-6623-49a5-9e54-2a5f53bb5e20 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:00.054897 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f03c71ad-6623-49a5-9e54-2a5f53bb5e20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:00.061479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1228/2463] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:12:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:00.074802 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c71ad-6623-49a5-9e54-2a5f53bb5e20 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:00.080312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0d4f9-98c1-498e-b43a-0090311a19f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:00.080971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d4f9-98c1-498e-b43a-0090311a19f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:00.081587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d4f9-98c1-498e-b43a-0090311a19f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:00.081587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d4f9-98c1-498e-b43a-0090311a19f2 admin admin] PATH_INFO: `/v3/users/a1bc5bf55e9d452b88757ea4080165e0` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:00.082912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0d4f9-98c1-498e-b43a-0090311a19f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1bc5bf55e9d452b88757ea4080165e0)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:00.084934 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a0d4f9-98c1-498e-b43a-0090311a19f2 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:00.359046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-97a0d4f9-98c1-498e-b43a-0090311a19f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1bc5bf55e9d452b88757ea4080165e0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:00.368874 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-97a0d4f9-98c1-498e-b43a-0090311a19f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1bc5bf55e9d452b88757ea4080165e0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:00.473104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-97a0d4f9-98c1-498e-b43a-0090311a19f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1bc5bf55e9d452b88757ea4080165e0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:00.488194 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-97a0d4f9-98c1-498e-b43a-0090311a19f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1bc5bf55e9d452b88757ea4080165e0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:00.489093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-97a0d4f9-98c1-498e-b43a-0090311a19f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1bc5bf55e9d452b88757ea4080165e0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:00.496375 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1236/2464] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:00 2022] DELETE /identity/v3/users/a1bc5bf55e9d452b88757ea4080165e0 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:00.807993 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75218a62-40c0-45a1-b27a-792a8d3db312 service placement] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:00.820083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75218a62-40c0-45a1-b27a-792a8d3db312 service placement] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:00.887076 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c518234-bb37-45db-96d2-45a12fc7b71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:00.904718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c518234-bb37-45db-96d2-45a12fc7b71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:00.904718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c518234-bb37-45db-96d2-45a12fc7b71b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:00.904718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c518234-bb37-45db-96d2-45a12fc7b71b admin admin] PATH_INFO: `/v3/projects/08db5f6552054a6c97ed644366d18b22` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:00.904718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c518234-bb37-45db-96d2-45a12fc7b71b admin admin] RBAC: Authorizing `identity:delete_project(project_id=08db5f6552054a6c97ed644366d18b22)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:00.904718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c518234-bb37-45db-96d2-45a12fc7b71b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:01.035200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-0c518234-bb37-45db-96d2-45a12fc7b71b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08db5f6552054a6c97ed644366d18b22', 'request_id': 'req-0c518234-bb37-45db-96d2-45a12fc7b71b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:01.042235 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-0c518234-bb37-45db-96d2-45a12fc7b71b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08db5f6552054a6c97ed644366d18b22', 'request_id': 'req-0c518234-bb37-45db-96d2-45a12fc7b71b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:01.049186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-0c518234-bb37-45db-96d2-45a12fc7b71b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08db5f6552054a6c97ed644366d18b22', 'request_id': 'req-0c518234-bb37-45db-96d2-45a12fc7b71b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:01.135766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-0c518234-bb37-45db-96d2-45a12fc7b71b admin admin] The token cache is being invalidate because project 08db5f6552054a6c97ed644366d18b22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:01.136032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-0c518234-bb37-45db-96d2-45a12fc7b71b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:01.136678 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-0c518234-bb37-45db-96d2-45a12fc7b71b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:01.138209 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1229/2465] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:00 2022] DELETE /identity/v3/projects/08db5f6552054a6c97ed644366d18b22 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:01.144492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601aaae6-98ff-46cb-a685-17bb6dbbd778 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:01.145022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601aaae6-98ff-46cb-a685-17bb6dbbd778 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:01.145475 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601aaae6-98ff-46cb-a685-17bb6dbbd778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:01.171419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-601aaae6-98ff-46cb-a685-17bb6dbbd778 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:01.172658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-601aaae6-98ff-46cb-a685-17bb6dbbd778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:01.235431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1237/2466] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:01.244717 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906f406-c4f8-4c34-9355-8bacd6606d26 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:01.244999 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906f406-c4f8-4c34-9355-8bacd6606d26 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:01.245275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906f406-c4f8-4c34-9355-8bacd6606d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:01.266738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-0906f406-c4f8-4c34-9355-8bacd6606d26 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:01.267976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0906f406-c4f8-4c34-9355-8bacd6606d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:01.276644 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1230/2467] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:01.281435 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601aaae6-98ff-46cb-a685-17bb6dbbd778 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:01.291174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601aaae6-98ff-46cb-a685-17bb6dbbd778 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:01.296138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638590d-932d-4d1b-bef2-6c457d85c86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:01.296732 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638590d-932d-4d1b-bef2-6c457d85c86d admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:01.297008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638590d-932d-4d1b-bef2-6c457d85c86d admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:01.297277 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638590d-932d-4d1b-bef2-6c457d85c86d admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:01.297963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638590d-932d-4d1b-bef2-6c457d85c86d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:13:01.298262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638590d-932d-4d1b-bef2-6c457d85c86d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:01.299937 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638590d-932d-4d1b-bef2-6c457d85c86d admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:01.312664 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1238/2468] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:13:01.317051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906f406-c4f8-4c34-9355-8bacd6606d26 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:01.327585 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906f406-c4f8-4c34-9355-8bacd6606d26 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:01.332376 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd1641-bc22-4f74-a0d2-828ee78d06f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:01.332990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd1641-bc22-4f74-a0d2-828ee78d06f6 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:01.333275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd1641-bc22-4f74-a0d2-828ee78d06f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:01.333551 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd1641-bc22-4f74-a0d2-828ee78d06f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:01.334258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd1641-bc22-4f74-a0d2-828ee78d06f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:01.336204 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd1641-bc22-4f74-a0d2-828ee78d06f6 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:01.348357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:01.452108 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1231/2469] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:01 2022] POST /identity/v3/projects => generated 405 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:01.456461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638590d-932d-4d1b-bef2-6c457d85c86d admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:01.467225 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638590d-932d-4d1b-bef2-6c457d85c86d admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:01.517660 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:01.524523 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0996b09-7ef1-4c6d-9094-b2b24370934e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:01.525145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0996b09-7ef1-4c6d-9094-b2b24370934e admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:01.525471 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0996b09-7ef1-4c6d-9094-b2b24370934e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:01.526365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0996b09-7ef1-4c6d-9094-b2b24370934e admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:01.527048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0996b09-7ef1-4c6d-9094-b2b24370934e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:01.529070 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0996b09-7ef1-4c6d-9094-b2b24370934e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:01.617076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1239/2470] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:01 2022] POST /identity/v3/users => generated 303 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:13:01.621948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd1641-bc22-4f74-a0d2-828ee78d06f6 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:01.632658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd1641-bc22-4f74-a0d2-828ee78d06f6 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:01.639778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadcf62-f972-4fe7-a1f8-2dc3eb3439cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:01.640439 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadcf62-f972-4fe7-a1f8-2dc3eb3439cc admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:01.640720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadcf62-f972-4fe7-a1f8-2dc3eb3439cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:01.640998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadcf62-f972-4fe7-a1f8-2dc3eb3439cc admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:01.641709 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadcf62-f972-4fe7-a1f8-2dc3eb3439cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:01.643356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadcf62-f972-4fe7-a1f8-2dc3eb3439cc admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:01.654541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1232/2471] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:01.658791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0996b09-7ef1-4c6d-9094-b2b24370934e admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:01.669561 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0996b09-7ef1-4c6d-9094-b2b24370934e admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:01.678679 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92876137-cd82-4454-b5cf-da0ebd0fcb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:01.682478 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92876137-cd82-4454-b5cf-da0ebd0fcb5c admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:01.682786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92876137-cd82-4454-b5cf-da0ebd0fcb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:01.683060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92876137-cd82-4454-b5cf-da0ebd0fcb5c admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:01.683786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92876137-cd82-4454-b5cf-da0ebd0fcb5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:01.685699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92876137-cd82-4454-b5cf-da0ebd0fcb5c admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:01.698286 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1240/2472] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:01.703882 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadcf62-f972-4fe7-a1f8-2dc3eb3439cc admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:01.714433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadcf62-f972-4fe7-a1f8-2dc3eb3439cc admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:01.719383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0274b-17b2-43c8-a474-4ff080e9916b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:01.720118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0274b-17b2-43c8-a474-4ff080e9916b admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:01.720443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0274b-17b2-43c8-a474-4ff080e9916b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:01.720763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0274b-17b2-43c8-a474-4ff080e9916b admin admin] PATH_INFO: `/v3/projects/1ff20b917af844768103d8d07ab5e953/users/1b3196831c2c475b80448cc3c97720ff/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:01.736534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0274b-17b2-43c8-a474-4ff080e9916b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ff20b917af844768103d8d07ab5e953, user_id=1b3196831c2c475b80448cc3c97720ff, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:01.739507 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0274b-17b2-43c8-a474-4ff080e9916b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:01.839365 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1233/2473] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:01 2022] PUT /identity/v3/projects/1ff20b917af844768103d8d07ab5e953/users/1b3196831c2c475b80448cc3c97720ff/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:01.844869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7789f6-8b1d-41bd-8dc3-20a3df4c7312 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:01.849406 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7789f6-8b1d-41bd-8dc3-20a3df4c7312 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:01.849406 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7789f6-8b1d-41bd-8dc3-20a3df4c7312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:01.866967 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-cf7789f6-8b1d-41bd-8dc3-20a3df4c7312 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:01.868248 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-cf7789f6-8b1d-41bd-8dc3-20a3df4c7312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:01.930380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1241/2474] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:13:01.940276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0274b-17b2-43c8-a474-4ff080e9916b admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:01.942053 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-bec0274b-17b2-43c8-a474-4ff080e9916b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:02.000808 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0274b-17b2-43c8-a474-4ff080e9916b admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:02.004481 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d1764-1d92-4a21-a423-bdc0a7abb889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:02.005093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d1764-1d92-4a21-a423-bdc0a7abb889 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:02.005373 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d1764-1d92-4a21-a423-bdc0a7abb889 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:02.005647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d1764-1d92-4a21-a423-bdc0a7abb889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:02.015591 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d1764-1d92-4a21-a423-bdc0a7abb889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:02.017017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d1764-1d92-4a21-a423-bdc0a7abb889 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:02.027819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1234/2475] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:03.498230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00138c5e-f27c-42e0-9ed3-c59bc3abda37 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:03.498754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00138c5e-f27c-42e0-9ed3-c59bc3abda37 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:03.498877 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00138c5e-f27c-42e0-9ed3-c59bc3abda37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:03.528868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-00138c5e-f27c-42e0-9ed3-c59bc3abda37 None None] MFA Rules not processed for user `1b3196831c2c475b80448cc3c97720ff`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:03.530076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-00138c5e-f27c-42e0-9ed3-c59bc3abda37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:03.604607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1242/2476] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:03 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:03.609055 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d1764-1d92-4a21-a423-bdc0a7abb889 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:03.622984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d1764-1d92-4a21-a423-bdc0a7abb889 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:03.628050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee37851-3e38-4ec2-bb1e-d5d8b906560c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:03.628718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee37851-3e38-4ec2-bb1e-d5d8b906560c admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:03.628990 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee37851-3e38-4ec2-bb1e-d5d8b906560c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:03.629269 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee37851-3e38-4ec2-bb1e-d5d8b906560c admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:03.629948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee37851-3e38-4ec2-bb1e-d5d8b906560c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:03.631969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee37851-3e38-4ec2-bb1e-d5d8b906560c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:03.721324 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1235/2477] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:03 2022] POST /identity/v3/projects => generated 403 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:03.725505 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00138c5e-f27c-42e0-9ed3-c59bc3abda37 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:03.737307 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00138c5e-f27c-42e0-9ed3-c59bc3abda37 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:03.796639 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a8691-0080-482f-9b03-fea4332abaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:03.797274 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a8691-0080-482f-9b03-fea4332abaed admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:03.797558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a8691-0080-482f-9b03-fea4332abaed admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:03.797835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a8691-0080-482f-9b03-fea4332abaed admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:03.798518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a8691-0080-482f-9b03-fea4332abaed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:03.800489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a8691-0080-482f-9b03-fea4332abaed admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:03.810878 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:03.936119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1243/2478] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:03 2022] POST /identity/v3/users => generated 308 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:03.937728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee37851-3e38-4ec2-bb1e-d5d8b906560c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:03.949771 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee37851-3e38-4ec2-bb1e-d5d8b906560c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:03.955010 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6504320-dbf8-4cd5-85de-3e11f6886fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:03.955664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6504320-dbf8-4cd5-85de-3e11f6886fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:03.955951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6504320-dbf8-4cd5-85de-3e11f6886fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:03.956230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6504320-dbf8-4cd5-85de-3e11f6886fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:03.956962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6504320-dbf8-4cd5-85de-3e11f6886fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:03.958578 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6504320-dbf8-4cd5-85de-3e11f6886fe0 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:03.970479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1236/2479] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:03.977055 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a8691-0080-482f-9b03-fea4332abaed admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:03.988653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a8691-0080-482f-9b03-fea4332abaed admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:03.993717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde32ef-d5e8-4fd4-a043-caf0cd3e933a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:03.995026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde32ef-d5e8-4fd4-a043-caf0cd3e933a admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:03.995429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde32ef-d5e8-4fd4-a043-caf0cd3e933a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:03.995771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde32ef-d5e8-4fd4-a043-caf0cd3e933a admin admin] PATH_INFO: `/v3/projects/ea6f4d9c358c407ba55969fa9c4f046d/users/950a7aaf7d9b46bea1df14b2051378cb/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:04.015170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde32ef-d5e8-4fd4-a043-caf0cd3e933a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea6f4d9c358c407ba55969fa9c4f046d, user_id=950a7aaf7d9b46bea1df14b2051378cb, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:04.017893 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde32ef-d5e8-4fd4-a043-caf0cd3e933a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:04.136700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1244/2480] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:03 2022] PUT /identity/v3/projects/ea6f4d9c358c407ba55969fa9c4f046d/users/950a7aaf7d9b46bea1df14b2051378cb/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:04.145040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6504320-dbf8-4cd5-85de-3e11f6886fe0 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:04.154187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6504320-dbf8-4cd5-85de-3e11f6886fe0 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:04.216094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07895f1-de18-4ab1-8350-1d1d2112ffd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:04.216720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07895f1-de18-4ab1-8350-1d1d2112ffd2 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:04.217002 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07895f1-de18-4ab1-8350-1d1d2112ffd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:04.217277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07895f1-de18-4ab1-8350-1d1d2112ffd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:04.217973 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07895f1-de18-4ab1-8350-1d1d2112ffd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:04.219603 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07895f1-de18-4ab1-8350-1d1d2112ffd2 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:04.231989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1237/2481] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:04.238129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde32ef-d5e8-4fd4-a043-caf0cd3e933a admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:04.268128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde32ef-d5e8-4fd4-a043-caf0cd3e933a admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:04.273176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8eda81-47bf-4c5b-a8aa-235fe4342605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:04.273906 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eda81-47bf-4c5b-a8aa-235fe4342605 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:04.274191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eda81-47bf-4c5b-a8aa-235fe4342605 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:04.274482 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eda81-47bf-4c5b-a8aa-235fe4342605 admin admin] PATH_INFO: `/v3/projects/ea6f4d9c358c407ba55969fa9c4f046d/users/950a7aaf7d9b46bea1df14b2051378cb/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:04.277442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8eda81-47bf-4c5b-a8aa-235fe4342605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea6f4d9c358c407ba55969fa9c4f046d, user_id=950a7aaf7d9b46bea1df14b2051378cb, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:04.280174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8eda81-47bf-4c5b-a8aa-235fe4342605 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:04.292258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1245/2482] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:04 2022] PUT /identity/v3/projects/ea6f4d9c358c407ba55969fa9c4f046d/users/950a7aaf7d9b46bea1df14b2051378cb/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:05.801082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da32b4-d648-4ea7-b35f-1eaa20637ab4 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:05.801661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da32b4-d648-4ea7-b35f-1eaa20637ab4 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:05.801661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da32b4-d648-4ea7-b35f-1eaa20637ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:05.840192 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-84da32b4-d648-4ea7-b35f-1eaa20637ab4 None None] MFA Rules not processed for user `950a7aaf7d9b46bea1df14b2051378cb`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:05.841447 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-84da32b4-d648-4ea7-b35f-1eaa20637ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:05.853206 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:05.937475 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1238/2483] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:05 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:07.621056 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8eda81-47bf-4c5b-a8aa-235fe4342605 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:07.730085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8eda81-47bf-4c5b-a8aa-235fe4342605 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:07.735868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433692f-f4fb-4784-a2f3-7d4aa515bcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:07.736708 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433692f-f4fb-4784-a2f3-7d4aa515bcb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:07.737245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433692f-f4fb-4784-a2f3-7d4aa515bcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:07.737521 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433692f-f4fb-4784-a2f3-7d4aa515bcb0 admin admin] PATH_INFO: `/v3/users/1b3196831c2c475b80448cc3c97720ff` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:07.738767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433692f-f4fb-4784-a2f3-7d4aa515bcb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b3196831c2c475b80448cc3c97720ff)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:07.740806 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433692f-f4fb-4784-a2f3-7d4aa515bcb0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:07.918543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1433692f-f4fb-4784-a2f3-7d4aa515bcb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b3196831c2c475b80448cc3c97720ff'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:07.925615 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1433692f-f4fb-4784-a2f3-7d4aa515bcb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b3196831c2c475b80448cc3c97720ff'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:07.998516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1433692f-f4fb-4784-a2f3-7d4aa515bcb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b3196831c2c475b80448cc3c97720ff'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:08.011811 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1433692f-f4fb-4784-a2f3-7d4aa515bcb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b3196831c2c475b80448cc3c97720ff'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:08.012590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1433692f-f4fb-4784-a2f3-7d4aa515bcb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b3196831c2c475b80448cc3c97720ff'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:08.019724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1246/2484] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:07 2022] DELETE /identity/v3/users/1b3196831c2c475b80448cc3c97720ff => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:08.269924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da32b4-d648-4ea7-b35f-1eaa20637ab4 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:08.283660 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da32b4-d648-4ea7-b35f-1eaa20637ab4 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:08.364535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:08.365206 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:08.365497 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:08.365768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] PATH_INFO: `/v3/projects/1ff20b917af844768103d8d07ab5e953` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:08.366972 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ff20b917af844768103d8d07ab5e953)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:08.368915 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:08.487287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ff20b917af844768103d8d07ab5e953', 'request_id': 'req-6c40ac82-a90f-45b1-9b3e-347982f09f7c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:08.495231 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ff20b917af844768103d8d07ab5e953', 'request_id': 'req-6c40ac82-a90f-45b1-9b3e-347982f09f7c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:08.502200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ff20b917af844768103d8d07ab5e953', 'request_id': 'req-6c40ac82-a90f-45b1-9b3e-347982f09f7c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:08.588971 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] The token cache is being invalidate because project 1ff20b917af844768103d8d07ab5e953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:08.589458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:08.590079 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:08.591790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1239/2485] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:08 2022] DELETE /identity/v3/projects/1ff20b917af844768103d8d07ab5e953 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:08.596137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433692f-f4fb-4784-a2f3-7d4aa515bcb0 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:08.597573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1433692f-f4fb-4784-a2f3-7d4aa515bcb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:08.610750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433692f-f4fb-4784-a2f3-7d4aa515bcb0 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:08.647819 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:08.675273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c5b99-0fac-434a-a51b-8992b579b221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:08.675855 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c5b99-0fac-434a-a51b-8992b579b221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:08.676108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c5b99-0fac-434a-a51b-8992b579b221 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:08.676376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c5b99-0fac-434a-a51b-8992b579b221 admin admin] PATH_INFO: `/v3/users/950a7aaf7d9b46bea1df14b2051378cb` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:08.677658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c5b99-0fac-434a-a51b-8992b579b221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=950a7aaf7d9b46bea1df14b2051378cb)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:08.682837 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c5b99-0fac-434a-a51b-8992b579b221 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:08.894893 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-404c5b99-0fac-434a-a51b-8992b579b221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '950a7aaf7d9b46bea1df14b2051378cb'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:08.902674 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-404c5b99-0fac-434a-a51b-8992b579b221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '950a7aaf7d9b46bea1df14b2051378cb'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:08.982112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-404c5b99-0fac-434a-a51b-8992b579b221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '950a7aaf7d9b46bea1df14b2051378cb'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:08.995933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-404c5b99-0fac-434a-a51b-8992b579b221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '950a7aaf7d9b46bea1df14b2051378cb'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:08.996653 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-404c5b99-0fac-434a-a51b-8992b579b221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '950a7aaf7d9b46bea1df14b2051378cb'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:09.002984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1247/2486] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:08 2022] DELETE /identity/v3/users/950a7aaf7d9b46bea1df14b2051378cb => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:09.296051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:09.296051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:09.296051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40ac82-a90f-45b1-9b3e-347982f09f7c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:09.335017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:09.359970 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bd9ae-5c7d-4723-9762-19f4e6c92c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:09.360611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bd9ae-5c7d-4723-9762-19f4e6c92c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:09.360894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bd9ae-5c7d-4723-9762-19f4e6c92c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:09.361165 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bd9ae-5c7d-4723-9762-19f4e6c92c33 admin admin] PATH_INFO: `/v3/projects/ea6f4d9c358c407ba55969fa9c4f046d` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:09.362691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bd9ae-5c7d-4723-9762-19f4e6c92c33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea6f4d9c358c407ba55969fa9c4f046d)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:09.364663 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bd9ae-5c7d-4723-9762-19f4e6c92c33 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:09.502298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-718bd9ae-5c7d-4723-9762-19f4e6c92c33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea6f4d9c358c407ba55969fa9c4f046d', 'request_id': 'req-718bd9ae-5c7d-4723-9762-19f4e6c92c33'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:09.508659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-718bd9ae-5c7d-4723-9762-19f4e6c92c33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea6f4d9c358c407ba55969fa9c4f046d', 'request_id': 'req-718bd9ae-5c7d-4723-9762-19f4e6c92c33'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:09.515505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-718bd9ae-5c7d-4723-9762-19f4e6c92c33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea6f4d9c358c407ba55969fa9c4f046d', 'request_id': 'req-718bd9ae-5c7d-4723-9762-19f4e6c92c33'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:09.604695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-718bd9ae-5c7d-4723-9762-19f4e6c92c33 admin admin] The token cache is being invalidate because project ea6f4d9c358c407ba55969fa9c4f046d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:09.604984 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-718bd9ae-5c7d-4723-9762-19f4e6c92c33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:09.605695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-718bd9ae-5c7d-4723-9762-19f4e6c92c33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:09.607278 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1240/2487] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:09 2022] DELETE /identity/v3/projects/ea6f4d9c358c407ba55969fa9c4f046d => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:09.612693 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a227e54-a2da-4022-9240-602b147deebe None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:09.612971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a227e54-a2da-4022-9240-602b147deebe None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:09.613241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a227e54-a2da-4022-9240-602b147deebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:09.636582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-9a227e54-a2da-4022-9240-602b147deebe None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:09.637797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9a227e54-a2da-4022-9240-602b147deebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:09.699332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1248/2488] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:09.709607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e5303-7860-4727-bdce-c3dc7f23e971 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:09.709898 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e5303-7860-4727-bdce-c3dc7f23e971 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:09.710180 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e5303-7860-4727-bdce-c3dc7f23e971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:09.731565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-2b0e5303-7860-4727-bdce-c3dc7f23e971 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:09.732805 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-2b0e5303-7860-4727-bdce-c3dc7f23e971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:09.741323 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1241/2489] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:09.747142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a227e54-a2da-4022-9240-602b147deebe None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:09.758788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a227e54-a2da-4022-9240-602b147deebe None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:09.764453 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba861725-7ae5-4a95-9253-69a685f0c1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:09.765100 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba861725-7ae5-4a95-9253-69a685f0c1ab admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:09.765382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba861725-7ae5-4a95-9253-69a685f0c1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:09.765658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba861725-7ae5-4a95-9253-69a685f0c1ab admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:09.766368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba861725-7ae5-4a95-9253-69a685f0c1ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:13:09.766682 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba861725-7ae5-4a95-9253-69a685f0c1ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:09.768498 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba861725-7ae5-4a95-9253-69a685f0c1ab admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:09.782743 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1249/2490] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:13:09.789446 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e5303-7860-4727-bdce-c3dc7f23e971 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:09.808891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e5303-7860-4727-bdce-c3dc7f23e971 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:09.814121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0154cb7-ffc8-426e-8a04-09f7e26f3cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:09.814752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0154cb7-ffc8-426e-8a04-09f7e26f3cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:09.815033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0154cb7-ffc8-426e-8a04-09f7e26f3cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:09.815304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0154cb7-ffc8-426e-8a04-09f7e26f3cd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:09.816044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0154cb7-ffc8-426e-8a04-09f7e26f3cd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:09.817970 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0154cb7-ffc8-426e-8a04-09f7e26f3cd4 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:09.977262 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1242/2491] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:09 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:13:09.982190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba861725-7ae5-4a95-9253-69a685f0c1ab admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:09.994692 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba861725-7ae5-4a95-9253-69a685f0c1ab admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:10.039735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:10.057897 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e9d5b-f2ed-48de-9c14-e14f938a9217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:10.058541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e9d5b-f2ed-48de-9c14-e14f938a9217 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:10.058823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e9d5b-f2ed-48de-9c14-e14f938a9217 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:10.059118 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e9d5b-f2ed-48de-9c14-e14f938a9217 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:10.059808 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e9d5b-f2ed-48de-9c14-e14f938a9217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:10.061759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e9d5b-f2ed-48de-9c14-e14f938a9217 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:10.148349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1250/2492] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:09 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:10.152643 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0154cb7-ffc8-426e-8a04-09f7e26f3cd4 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:10.163345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0154cb7-ffc8-426e-8a04-09f7e26f3cd4 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:10.168938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11642253-55f6-4090-a8e2-0bb2e3434ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:10.169561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11642253-55f6-4090-a8e2-0bb2e3434ada admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:10.169850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11642253-55f6-4090-a8e2-0bb2e3434ada admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:10.170118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11642253-55f6-4090-a8e2-0bb2e3434ada admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:10.170821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11642253-55f6-4090-a8e2-0bb2e3434ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:10.172455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11642253-55f6-4090-a8e2-0bb2e3434ada admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:10.184038 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1243/2493] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:13:10.188254 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e9d5b-f2ed-48de-9c14-e14f938a9217 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:10.199317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e9d5b-f2ed-48de-9c14-e14f938a9217 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:10.204199 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95400e-773b-4891-91e7-b62e75571fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:10.204832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95400e-773b-4891-91e7-b62e75571fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:10.205110 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95400e-773b-4891-91e7-b62e75571fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:10.205383 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95400e-773b-4891-91e7-b62e75571fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:10.206149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95400e-773b-4891-91e7-b62e75571fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:10.207783 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95400e-773b-4891-91e7-b62e75571fb2 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:10.219645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1251/2494] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:10.224410 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11642253-55f6-4090-a8e2-0bb2e3434ada admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:10.235203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11642253-55f6-4090-a8e2-0bb2e3434ada admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:10.240181 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478524e6-dbbe-4dc2-863a-9408d81efb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:10.240857 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478524e6-dbbe-4dc2-863a-9408d81efb96 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:10.241142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478524e6-dbbe-4dc2-863a-9408d81efb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:10.241420 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478524e6-dbbe-4dc2-863a-9408d81efb96 admin admin] PATH_INFO: `/v3/projects/9fab9e42097f48bab65ff504ec1d6e60/users/770b3c9d279a40a9b53032f534b5fc3e/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:10.258264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478524e6-dbbe-4dc2-863a-9408d81efb96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fab9e42097f48bab65ff504ec1d6e60, user_id=770b3c9d279a40a9b53032f534b5fc3e, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:10.262227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478524e6-dbbe-4dc2-863a-9408d81efb96 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:10.365593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1244/2495] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:10 2022] PUT /identity/v3/projects/9fab9e42097f48bab65ff504ec1d6e60/users/770b3c9d279a40a9b53032f534b5fc3e/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:10.377973 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21414d7-5811-4352-8b54-0890043a14bc None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:10.378264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21414d7-5811-4352-8b54-0890043a14bc None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:10.378535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21414d7-5811-4352-8b54-0890043a14bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:10.408421 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b21414d7-5811-4352-8b54-0890043a14bc None None] MFA Rules not processed for user `770b3c9d279a40a9b53032f534b5fc3e`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:10.409638 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b21414d7-5811-4352-8b54-0890043a14bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:10.475554 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1252/2496] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:10.481259 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478524e6-dbbe-4dc2-863a-9408d81efb96 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:10.562106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478524e6-dbbe-4dc2-863a-9408d81efb96 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:10.566907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07dcaf-5e99-40c8-b191-f7dcb51c7e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:10.568361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07dcaf-5e99-40c8-b191-f7dcb51c7e9a admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:10.568361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07dcaf-5e99-40c8-b191-f7dcb51c7e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:10.568361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07dcaf-5e99-40c8-b191-f7dcb51c7e9a admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:10.568816 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07dcaf-5e99-40c8-b191-f7dcb51c7e9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:10.570772 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07dcaf-5e99-40c8-b191-f7dcb51c7e9a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:10.581451 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:10.675819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1245/2497] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:10 2022] POST /identity/v3/projects => generated 387 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:10.680115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21414d7-5811-4352-8b54-0890043a14bc None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:10.691965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21414d7-5811-4352-8b54-0890043a14bc None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:10.756105 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b3d6e-02ad-411c-a32b-79b5355ece59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:10.756828 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b3d6e-02ad-411c-a32b-79b5355ece59 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:10.757003 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b3d6e-02ad-411c-a32b-79b5355ece59 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:10.757272 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b3d6e-02ad-411c-a32b-79b5355ece59 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:10.757981 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b3d6e-02ad-411c-a32b-79b5355ece59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:10.760134 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b3d6e-02ad-411c-a32b-79b5355ece59 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:10.863031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1253/2498] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:10 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:10.867271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07dcaf-5e99-40c8-b191-f7dcb51c7e9a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:10.881809 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07dcaf-5e99-40c8-b191-f7dcb51c7e9a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:10.887928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc19cf0-4895-47e8-baf6-9823d252aad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:10.888691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc19cf0-4895-47e8-baf6-9823d252aad7 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:10.888972 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc19cf0-4895-47e8-baf6-9823d252aad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:10.889243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc19cf0-4895-47e8-baf6-9823d252aad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:10.890014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc19cf0-4895-47e8-baf6-9823d252aad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:10.891695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc19cf0-4895-47e8-baf6-9823d252aad7 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:10.905359 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1246/2499] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:10.909883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b3d6e-02ad-411c-a32b-79b5355ece59 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:10.921457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b3d6e-02ad-411c-a32b-79b5355ece59 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:10.926685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87dbb2-4ce5-4bef-b4fd-9dac0d1725d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:10.927140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87dbb2-4ce5-4bef-b4fd-9dac0d1725d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:10.927707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87dbb2-4ce5-4bef-b4fd-9dac0d1725d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:10.927984 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87dbb2-4ce5-4bef-b4fd-9dac0d1725d9 admin admin] PATH_INFO: `/v3/projects/67ffcc64e1d24c829dfa21a54165f8e5/users/f520ebf64326470994b943d7347c4af3/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:10.948180 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87dbb2-4ce5-4bef-b4fd-9dac0d1725d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ffcc64e1d24c829dfa21a54165f8e5, user_id=f520ebf64326470994b943d7347c4af3, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:10.950849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87dbb2-4ce5-4bef-b4fd-9dac0d1725d9 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:11.021996 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1254/2500] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:10 2022] PUT /identity/v3/projects/67ffcc64e1d24c829dfa21a54165f8e5/users/f520ebf64326470994b943d7347c4af3/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:11.026503 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc19cf0-4895-47e8-baf6-9823d252aad7 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:11.037989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc19cf0-4895-47e8-baf6-9823d252aad7 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:11.136569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0d198-67d1-40c0-9db0-7d1ee58cfb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:11.137261 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d198-67d1-40c0-9db0-7d1ee58cfb50 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:11.137552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d198-67d1-40c0-9db0-7d1ee58cfb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:11.137862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d198-67d1-40c0-9db0-7d1ee58cfb50 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:11.138546 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0d198-67d1-40c0-9db0-7d1ee58cfb50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:11.140207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0d198-67d1-40c0-9db0-7d1ee58cfb50 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:11.154742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1247/2501] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:11.158938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87dbb2-4ce5-4bef-b4fd-9dac0d1725d9 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:11.173399 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87dbb2-4ce5-4bef-b4fd-9dac0d1725d9 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:11.180217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e38186-c6b8-482e-bb8a-6f2306e4e70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:11.180814 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e38186-c6b8-482e-bb8a-6f2306e4e70f admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:11.181095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e38186-c6b8-482e-bb8a-6f2306e4e70f admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:11.181365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e38186-c6b8-482e-bb8a-6f2306e4e70f admin admin] PATH_INFO: `/v3/projects/67ffcc64e1d24c829dfa21a54165f8e5/users/f520ebf64326470994b943d7347c4af3/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:11.187452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e38186-c6b8-482e-bb8a-6f2306e4e70f admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ffcc64e1d24c829dfa21a54165f8e5, user_id=f520ebf64326470994b943d7347c4af3, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:11.190350 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e38186-c6b8-482e-bb8a-6f2306e4e70f admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:11.220262 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1255/2502] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:11 2022] PUT /identity/v3/projects/67ffcc64e1d24c829dfa21a54165f8e5/users/f520ebf64326470994b943d7347c4af3/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:11.232834 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3a535-dd21-407e-ae58-a3296aa08617 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:11.233758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3a535-dd21-407e-ae58-a3296aa08617 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:11.234097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3a535-dd21-407e-ae58-a3296aa08617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:11.284144 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-c6f3a535-dd21-407e-ae58-a3296aa08617 None None] MFA Rules not processed for user `f520ebf64326470994b943d7347c4af3`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:11.285791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c6f3a535-dd21-407e-ae58-a3296aa08617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:11.390607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1248/2503] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:11.398377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e38186-c6b8-482e-bb8a-6f2306e4e70f admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:11.400008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-98e38186-c6b8-482e-bb8a-6f2306e4e70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:11.406153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:11.493827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e38186-c6b8-482e-bb8a-6f2306e4e70f admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:11.500440 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24616d5-1fad-4669-a9a9-7ba45e542160 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:11.501635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24616d5-1fad-4669-a9a9-7ba45e542160 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:11.502124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24616d5-1fad-4669-a9a9-7ba45e542160 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:11.502590 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24616d5-1fad-4669-a9a9-7ba45e542160 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:11.518694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24616d5-1fad-4669-a9a9-7ba45e542160 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:11.521398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24616d5-1fad-4669-a9a9-7ba45e542160 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:11.538942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1256/2504] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:11.570385 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3a535-dd21-407e-ae58-a3296aa08617 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:11.659794 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3a535-dd21-407e-ae58-a3296aa08617 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:11.669225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05daf7c6-36c2-4dd2-b026-fe8ad7f8b2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:11.670625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05daf7c6-36c2-4dd2-b026-fe8ad7f8b2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:11.671097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05daf7c6-36c2-4dd2-b026-fe8ad7f8b2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:11.671703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05daf7c6-36c2-4dd2-b026-fe8ad7f8b2ff admin admin] PATH_INFO: `/v3/users/770b3c9d279a40a9b53032f534b5fc3e` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:11.673880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05daf7c6-36c2-4dd2-b026-fe8ad7f8b2ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=770b3c9d279a40a9b53032f534b5fc3e)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:11.677733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05daf7c6-36c2-4dd2-b026-fe8ad7f8b2ff admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:11.680618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:11.940090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-05daf7c6-36c2-4dd2-b026-fe8ad7f8b2ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770b3c9d279a40a9b53032f534b5fc3e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:11.940837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-05daf7c6-36c2-4dd2-b026-fe8ad7f8b2ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770b3c9d279a40a9b53032f534b5fc3e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:11.941486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-05daf7c6-36c2-4dd2-b026-fe8ad7f8b2ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770b3c9d279a40a9b53032f534b5fc3e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:11.950763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-05daf7c6-36c2-4dd2-b026-fe8ad7f8b2ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770b3c9d279a40a9b53032f534b5fc3e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:11.970062 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-05daf7c6-36c2-4dd2-b026-fe8ad7f8b2ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770b3c9d279a40a9b53032f534b5fc3e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:12.097029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1249/2505] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:11 2022] DELETE /identity/v3/users/770b3c9d279a40a9b53032f534b5fc3e => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:12.101404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286dd584-64ab-4b65-b086-8f1915538994 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:12.101985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286dd584-64ab-4b65-b086-8f1915538994 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:12.102096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286dd584-64ab-4b65-b086-8f1915538994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:12.126412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-286dd584-64ab-4b65-b086-8f1915538994 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:12.127493 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-286dd584-64ab-4b65-b086-8f1915538994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:12.211958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1257/2506] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:12.227517 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05daf7c6-36c2-4dd2-b026-fe8ad7f8b2ff admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:12.229134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-05daf7c6-36c2-4dd2-b026-fe8ad7f8b2ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:12.311922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05daf7c6-36c2-4dd2-b026-fe8ad7f8b2ff admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:12.315058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d3032-aa79-42b6-9391-7eb6cf0459fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:12.319617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d3032-aa79-42b6-9391-7eb6cf0459fa service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:12.319617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d3032-aa79-42b6-9391-7eb6cf0459fa service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:12.319617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d3032-aa79-42b6-9391-7eb6cf0459fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:12.336215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d3032-aa79-42b6-9391-7eb6cf0459fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:12.338432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d3032-aa79-42b6-9391-7eb6cf0459fa service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:12.350417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1250/2507] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:12.887844 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286dd584-64ab-4b65-b086-8f1915538994 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:12.888069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-286dd584-64ab-4b65-b086-8f1915538994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:12.903681 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286dd584-64ab-4b65-b086-8f1915538994 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:12.909089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:12.909788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:12.910492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:12.910492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] PATH_INFO: `/v3/projects/9fab9e42097f48bab65ff504ec1d6e60` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:12.911530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fab9e42097f48bab65ff504ec1d6e60)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:12.916925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:13.100786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fab9e42097f48bab65ff504ec1d6e60', 'request_id': 'req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:13.107884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fab9e42097f48bab65ff504ec1d6e60', 'request_id': 'req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:13.204240 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fab9e42097f48bab65ff504ec1d6e60', 'request_id': 'req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:13.217680 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] The token cache is being invalidate because project 9fab9e42097f48bab65ff504ec1d6e60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:13.217950 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:13.218683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:13.220276 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1258/2508] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:12 2022] DELETE /identity/v3/projects/9fab9e42097f48bab65ff504ec1d6e60 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:13.224537 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d3032-aa79-42b6-9391-7eb6cf0459fa service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:13.235486 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d3032-aa79-42b6-9391-7eb6cf0459fa service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:13.310620 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2cbdb-1ff5-490e-9345-17583f130b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:13.315826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2cbdb-1ff5-490e-9345-17583f130b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:13.315826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2cbdb-1ff5-490e-9345-17583f130b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:13.315826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2cbdb-1ff5-490e-9345-17583f130b58 admin admin] PATH_INFO: `/v3/users/f520ebf64326470994b943d7347c4af3` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:13.315826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2cbdb-1ff5-490e-9345-17583f130b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f520ebf64326470994b943d7347c4af3)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:13.315826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2cbdb-1ff5-490e-9345-17583f130b58 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:13.401537 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:13.583940 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-41d2cbdb-1ff5-490e-9345-17583f130b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f520ebf64326470994b943d7347c4af3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:13.586032 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-41d2cbdb-1ff5-490e-9345-17583f130b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f520ebf64326470994b943d7347c4af3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:13.586892 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-41d2cbdb-1ff5-490e-9345-17583f130b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f520ebf64326470994b943d7347c4af3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:13.595544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-41d2cbdb-1ff5-490e-9345-17583f130b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f520ebf64326470994b943d7347c4af3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:13.608733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-41d2cbdb-1ff5-490e-9345-17583f130b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f520ebf64326470994b943d7347c4af3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:13.698912 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1251/2509] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:13 2022] DELETE /identity/v3/users/f520ebf64326470994b943d7347c4af3 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:14.481756 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:14.484084 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:14.504831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d11ac2-9fc3-471f-a3c7-cd4ff774b46f admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:14.536894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:14.578457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea282e2-0ac1-4272-88c7-80dc2886c95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:14.579119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea282e2-0ac1-4272-88c7-80dc2886c95c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:14.579481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea282e2-0ac1-4272-88c7-80dc2886c95c admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:14.579917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea282e2-0ac1-4272-88c7-80dc2886c95c admin admin] PATH_INFO: `/v3/projects/67ffcc64e1d24c829dfa21a54165f8e5` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:14.581184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea282e2-0ac1-4272-88c7-80dc2886c95c admin admin] RBAC: Authorizing `identity:delete_project(project_id=67ffcc64e1d24c829dfa21a54165f8e5)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:14.583113 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea282e2-0ac1-4272-88c7-80dc2886c95c admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:14.740693 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-fea282e2-0ac1-4272-88c7-80dc2886c95c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67ffcc64e1d24c829dfa21a54165f8e5', 'request_id': 'req-fea282e2-0ac1-4272-88c7-80dc2886c95c'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:14.749093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-fea282e2-0ac1-4272-88c7-80dc2886c95c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67ffcc64e1d24c829dfa21a54165f8e5', 'request_id': 'req-fea282e2-0ac1-4272-88c7-80dc2886c95c'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:14.839346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-fea282e2-0ac1-4272-88c7-80dc2886c95c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67ffcc64e1d24c829dfa21a54165f8e5', 'request_id': 'req-fea282e2-0ac1-4272-88c7-80dc2886c95c'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:14.852171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-fea282e2-0ac1-4272-88c7-80dc2886c95c admin admin] The token cache is being invalidate because project 67ffcc64e1d24c829dfa21a54165f8e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:14.852574 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-fea282e2-0ac1-4272-88c7-80dc2886c95c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:14.853107 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-fea282e2-0ac1-4272-88c7-80dc2886c95c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:14.855896 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1259/2510] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:14 2022] DELETE /identity/v3/projects/67ffcc64e1d24c829dfa21a54165f8e5 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:14.861142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61700ec-2321-4496-9a94-7f8071f6d83a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:14.861685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61700ec-2321-4496-9a94-7f8071f6d83a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:14.861685 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61700ec-2321-4496-9a94-7f8071f6d83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:14.884796 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-e61700ec-2321-4496-9a94-7f8071f6d83a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:14.886913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e61700ec-2321-4496-9a94-7f8071f6d83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:14.959645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1252/2511] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:14.967823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7324ebe-34f5-434e-9e06-7252847a7584 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:14.968109 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7324ebe-34f5-434e-9e06-7252847a7584 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:14.968440 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7324ebe-34f5-434e-9e06-7252847a7584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:14.992600 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-f7324ebe-34f5-434e-9e06-7252847a7584 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:14.993871 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f7324ebe-34f5-434e-9e06-7252847a7584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:15.004836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1260/2512] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:13:15.010415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61700ec-2321-4496-9a94-7f8071f6d83a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:15.027454 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61700ec-2321-4496-9a94-7f8071f6d83a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:15.034179 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f4153-2fb6-4dbd-b64b-8d87446d8a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:15.034811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f4153-2fb6-4dbd-b64b-8d87446d8a7a admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:15.035097 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f4153-2fb6-4dbd-b64b-8d87446d8a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:15.035369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f4153-2fb6-4dbd-b64b-8d87446d8a7a admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:15.036098 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f4153-2fb6-4dbd-b64b-8d87446d8a7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:13:15.047225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f4153-2fb6-4dbd-b64b-8d87446d8a7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:15.047225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f4153-2fb6-4dbd-b64b-8d87446d8a7a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:15.054297 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1253/2513] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:13:15.059210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7324ebe-34f5-434e-9e06-7252847a7584 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:15.076917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7324ebe-34f5-434e-9e06-7252847a7584 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:15.076917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ee86c-a1bb-4dc8-ae58-5926c33b6d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:15.076917 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ee86c-a1bb-4dc8-ae58-5926c33b6d53 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:15.077342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ee86c-a1bb-4dc8-ae58-5926c33b6d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:15.077342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ee86c-a1bb-4dc8-ae58-5926c33b6d53 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:15.078037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ee86c-a1bb-4dc8-ae58-5926c33b6d53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:15.079989 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ee86c-a1bb-4dc8-ae58-5926c33b6d53 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:15.175121 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1261/2514] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:15 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:13:15.179834 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f4153-2fb6-4dbd-b64b-8d87446d8a7a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:15.191806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f4153-2fb6-4dbd-b64b-8d87446d8a7a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:15.262484 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ccfac-00e6-48d5-851c-b33bd958b1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:15.263124 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ccfac-00e6-48d5-851c-b33bd958b1e9 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:15.263415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ccfac-00e6-48d5-851c-b33bd958b1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:15.263953 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ccfac-00e6-48d5-851c-b33bd958b1e9 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:15.264664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ccfac-00e6-48d5-851c-b33bd958b1e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:15.266805 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ccfac-00e6-48d5-851c-b33bd958b1e9 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:15.435413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1254/2515] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:15 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:15.440958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ee86c-a1bb-4dc8-ae58-5926c33b6d53 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:15.452441 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ee86c-a1bb-4dc8-ae58-5926c33b6d53 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:15.457575 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e2ac0-d19d-4ec5-8dd1-f43fccc9aacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:15.458173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e2ac0-d19d-4ec5-8dd1-f43fccc9aacc admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:15.458501 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e2ac0-d19d-4ec5-8dd1-f43fccc9aacc admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:15.458847 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e2ac0-d19d-4ec5-8dd1-f43fccc9aacc admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:15.459542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e2ac0-d19d-4ec5-8dd1-f43fccc9aacc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:15.461201 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e2ac0-d19d-4ec5-8dd1-f43fccc9aacc admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:15.473585 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1262/2516] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:15.482228 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ccfac-00e6-48d5-851c-b33bd958b1e9 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:15.492757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ccfac-00e6-48d5-851c-b33bd958b1e9 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:15.497806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ea4b3-805b-4362-a08d-c18dfbac61ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:15.498433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ea4b3-805b-4362-a08d-c18dfbac61ed admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:15.498713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ea4b3-805b-4362-a08d-c18dfbac61ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:15.498988 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ea4b3-805b-4362-a08d-c18dfbac61ed admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:15.499761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ea4b3-805b-4362-a08d-c18dfbac61ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:15.504747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ea4b3-805b-4362-a08d-c18dfbac61ed admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:15.521523 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1255/2517] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:15.526630 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e2ac0-d19d-4ec5-8dd1-f43fccc9aacc admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:15.537332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e2ac0-d19d-4ec5-8dd1-f43fccc9aacc admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:15.542992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6e921-c32b-4a35-b2c0-4745f6277f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:15.543708 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6e921-c32b-4a35-b2c0-4745f6277f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:15.543993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6e921-c32b-4a35-b2c0-4745f6277f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:15.544265 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6e921-c32b-4a35-b2c0-4745f6277f20 admin admin] PATH_INFO: `/v3/projects/bc3746541d5b4564a45a2aa9365b797c/users/bdecdbc3bcaf412494e776985112b359/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:15.562143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6e921-c32b-4a35-b2c0-4745f6277f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc3746541d5b4564a45a2aa9365b797c, user_id=bdecdbc3bcaf412494e776985112b359, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:15.565285 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6e921-c32b-4a35-b2c0-4745f6277f20 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:15.683664 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1263/2518] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:15 2022] PUT /identity/v3/projects/bc3746541d5b4564a45a2aa9365b797c/users/bdecdbc3bcaf412494e776985112b359/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:15.702570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d6e4a-5755-4b47-8938-2c3eb931b236 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:15.705607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d6e4a-5755-4b47-8938-2c3eb931b236 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:15.705607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d6e4a-5755-4b47-8938-2c3eb931b236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:15.726160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:15.750547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ce9d6e4a-5755-4b47-8938-2c3eb931b236 None None] MFA Rules not processed for user `bdecdbc3bcaf412494e776985112b359`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:15.751938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ce9d6e4a-5755-4b47-8938-2c3eb931b236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:15.858947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1256/2519] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:15.864980 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6e921-c32b-4a35-b2c0-4745f6277f20 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:15.933889 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:15.939722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6e921-c32b-4a35-b2c0-4745f6277f20 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:15.944858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e208a9a-3629-44c5-9d34-cc58f6973990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:15.950219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e208a9a-3629-44c5-9d34-cc58f6973990 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:15.950219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e208a9a-3629-44c5-9d34-cc58f6973990 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:15.950219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e208a9a-3629-44c5-9d34-cc58f6973990 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:15.950219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e208a9a-3629-44c5-9d34-cc58f6973990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:15.950219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e208a9a-3629-44c5-9d34-cc58f6973990 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:16.072163 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1264/2520] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:15 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:16.073541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d6e4a-5755-4b47-8938-2c3eb931b236 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:16.089019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d6e4a-5755-4b47-8938-2c3eb931b236 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:16.213474 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a811742-2bcc-40c0-af58-0594c2eb7c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:16.214719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a811742-2bcc-40c0-af58-0594c2eb7c75 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:16.215210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a811742-2bcc-40c0-af58-0594c2eb7c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:16.215803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a811742-2bcc-40c0-af58-0594c2eb7c75 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:16.217030 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a811742-2bcc-40c0-af58-0594c2eb7c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:16.220686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a811742-2bcc-40c0-af58-0594c2eb7c75 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:16.347084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1257/2521] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:16 2022] POST /identity/v3/users => generated 299 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:16.351202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e208a9a-3629-44c5-9d34-cc58f6973990 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:16.363365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e208a9a-3629-44c5-9d34-cc58f6973990 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:16.369738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54f310-306c-4cba-86eb-33e9bd75c344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:16.370451 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54f310-306c-4cba-86eb-33e9bd75c344 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:16.370775 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54f310-306c-4cba-86eb-33e9bd75c344 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:16.371057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54f310-306c-4cba-86eb-33e9bd75c344 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:16.371804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54f310-306c-4cba-86eb-33e9bd75c344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:16.373431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54f310-306c-4cba-86eb-33e9bd75c344 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:16.385810 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1265/2522] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:16.390736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a811742-2bcc-40c0-af58-0594c2eb7c75 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:16.402835 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a811742-2bcc-40c0-af58-0594c2eb7c75 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:16.407915 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bebbe4-70fa-4d6f-969b-df7064f22522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:16.408648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bebbe4-70fa-4d6f-969b-df7064f22522 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:16.408934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bebbe4-70fa-4d6f-969b-df7064f22522 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:16.409215 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bebbe4-70fa-4d6f-969b-df7064f22522 admin admin] PATH_INFO: `/v3/projects/fdf82c0429824d86a923161df5d2004d/users/24e4590a76d84a40822d78ca86028d06/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:16.428410 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bebbe4-70fa-4d6f-969b-df7064f22522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf82c0429824d86a923161df5d2004d, user_id=24e4590a76d84a40822d78ca86028d06, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:16.436491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bebbe4-70fa-4d6f-969b-df7064f22522 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:16.594545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1258/2523] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:16 2022] PUT /identity/v3/projects/fdf82c0429824d86a923161df5d2004d/users/24e4590a76d84a40822d78ca86028d06/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:16.598092 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54f310-306c-4cba-86eb-33e9bd75c344 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:16.617451 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54f310-306c-4cba-86eb-33e9bd75c344 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:16.686763 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789d972-9c12-41bf-b8d6-d20736d43475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:16.687403 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789d972-9c12-41bf-b8d6-d20736d43475 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:16.687722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789d972-9c12-41bf-b8d6-d20736d43475 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:16.687999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789d972-9c12-41bf-b8d6-d20736d43475 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:16.688715 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789d972-9c12-41bf-b8d6-d20736d43475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:16.690369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789d972-9c12-41bf-b8d6-d20736d43475 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:16.703153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1266/2524] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:16.708040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bebbe4-70fa-4d6f-969b-df7064f22522 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:16.719736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bebbe4-70fa-4d6f-969b-df7064f22522 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:16.725167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f369f3f-f9f8-4e07-a1ad-c7fd311216f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:16.725903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f369f3f-f9f8-4e07-a1ad-c7fd311216f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:16.726190 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f369f3f-f9f8-4e07-a1ad-c7fd311216f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:16.726473 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f369f3f-f9f8-4e07-a1ad-c7fd311216f9 admin admin] PATH_INFO: `/v3/projects/fdf82c0429824d86a923161df5d2004d/users/24e4590a76d84a40822d78ca86028d06/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:16.728800 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f369f3f-f9f8-4e07-a1ad-c7fd311216f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf82c0429824d86a923161df5d2004d, user_id=24e4590a76d84a40822d78ca86028d06, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:16.731676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f369f3f-f9f8-4e07-a1ad-c7fd311216f9 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:16.744725 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1259/2525] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:16 2022] PUT /identity/v3/projects/fdf82c0429824d86a923161df5d2004d/users/24e4590a76d84a40822d78ca86028d06/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:16.756893 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caae391-7d30-44f2-bb21-74f100c6683d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:16.757266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caae391-7d30-44f2-bb21-74f100c6683d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:16.757546 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caae391-7d30-44f2-bb21-74f100c6683d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:16.794816 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-1caae391-7d30-44f2-bb21-74f100c6683d None None] MFA Rules not processed for user `24e4590a76d84a40822d78ca86028d06`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:16.796072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1caae391-7d30-44f2-bb21-74f100c6683d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:16.871130 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1267/2526] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:16.884156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f369f3f-f9f8-4e07-a1ad-c7fd311216f9 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:16.886428 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-5f369f3f-f9f8-4e07-a1ad-c7fd311216f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:16.960434 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:16.966313 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f369f3f-f9f8-4e07-a1ad-c7fd311216f9 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:16.970575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09a074-7575-4c66-9672-98c483dcc0fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:16.971253 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09a074-7575-4c66-9672-98c483dcc0fb service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:16.971575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09a074-7575-4c66-9672-98c483dcc0fb service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:16.972986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09a074-7575-4c66-9672-98c483dcc0fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:16.982622 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09a074-7575-4c66-9672-98c483dcc0fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:16.984171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09a074-7575-4c66-9672-98c483dcc0fb service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:16.995883 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1260/2527] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:13:17.071096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caae391-7d30-44f2-bb21-74f100c6683d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:17.148221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:17.162360 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caae391-7d30-44f2-bb21-74f100c6683d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:17.171898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78c933-d6eb-4ee4-bac3-255d9ccd506a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:17.172560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78c933-d6eb-4ee4-bac3-255d9ccd506a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:17.172840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78c933-d6eb-4ee4-bac3-255d9ccd506a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:17.173114 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78c933-d6eb-4ee4-bac3-255d9ccd506a admin admin] PATH_INFO: `/v3/users/bdecdbc3bcaf412494e776985112b359` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:17.174319 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78c933-d6eb-4ee4-bac3-255d9ccd506a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdecdbc3bcaf412494e776985112b359)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:17.176348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78c933-d6eb-4ee4-bac3-255d9ccd506a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:17.436994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-9d78c933-d6eb-4ee4-bac3-255d9ccd506a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdecdbc3bcaf412494e776985112b359'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:17.444878 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-9d78c933-d6eb-4ee4-bac3-255d9ccd506a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdecdbc3bcaf412494e776985112b359'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:17.564352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-9d78c933-d6eb-4ee4-bac3-255d9ccd506a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdecdbc3bcaf412494e776985112b359'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:17.587440 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-9d78c933-d6eb-4ee4-bac3-255d9ccd506a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdecdbc3bcaf412494e776985112b359'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:17.588516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-9d78c933-d6eb-4ee4-bac3-255d9ccd506a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdecdbc3bcaf412494e776985112b359'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:17.596059 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1268/2528] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:17 2022] DELETE /identity/v3/users/bdecdbc3bcaf412494e776985112b359 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:17.601727 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f946e-8884-4364-a213-c7de420a0bb3 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:17.602647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f946e-8884-4364-a213-c7de420a0bb3 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:17.603137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f946e-8884-4364-a213-c7de420a0bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:17.624983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-4c1f946e-8884-4364-a213-c7de420a0bb3 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:17.626223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4c1f946e-8884-4364-a213-c7de420a0bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:17.709202 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1261/2529] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:13:17.721151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78c933-d6eb-4ee4-bac3-255d9ccd506a admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:17.723723 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-9d78c933-d6eb-4ee4-bac3-255d9ccd506a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:17.821928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78c933-d6eb-4ee4-bac3-255d9ccd506a admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:17.837687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e2b28-fd02-4d95-a2ca-3f471e6431d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:17.837687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e2b28-fd02-4d95-a2ca-3f471e6431d1 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:17.837687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e2b28-fd02-4d95-a2ca-3f471e6431d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:17.837687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e2b28-fd02-4d95-a2ca-3f471e6431d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:17.850988 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e2b28-fd02-4d95-a2ca-3f471e6431d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:17.854438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e2b28-fd02-4d95-a2ca-3f471e6431d1 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:17.872433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1269/2530] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:13:18.485741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f946e-8884-4364-a213-c7de420a0bb3 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:18.487791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4c1f946e-8884-4364-a213-c7de420a0bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:18.510144 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f946e-8884-4364-a213-c7de420a0bb3 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:18.515704 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:18.516611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:18.516825 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:18.517146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] PATH_INFO: `/v3/projects/bc3746541d5b4564a45a2aa9365b797c` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:18.518598 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc3746541d5b4564a45a2aa9365b797c)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:18.520774 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:18.746697 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc3746541d5b4564a45a2aa9365b797c', 'request_id': 'req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:18.753570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc3746541d5b4564a45a2aa9365b797c', 'request_id': 'req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:18.760726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc3746541d5b4564a45a2aa9365b797c', 'request_id': 'req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:18.849634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] The token cache is being invalidate because project bc3746541d5b4564a45a2aa9365b797c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:18.849942 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:18.850566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:18.852138 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1262/2531] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:18 2022] DELETE /identity/v3/projects/bc3746541d5b4564a45a2aa9365b797c => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:18.856195 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e2b28-fd02-4d95-a2ca-3f471e6431d1 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:18.866383 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e2b28-fd02-4d95-a2ca-3f471e6431d1 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:18.925621 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879365f7-1879-42bf-b3df-864cff303fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:18.926481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879365f7-1879-42bf-b3df-864cff303fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:18.926805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879365f7-1879-42bf-b3df-864cff303fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:18.927136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879365f7-1879-42bf-b3df-864cff303fc9 admin admin] PATH_INFO: `/v3/users/24e4590a76d84a40822d78ca86028d06` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:18.928612 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879365f7-1879-42bf-b3df-864cff303fc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24e4590a76d84a40822d78ca86028d06)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:18.931082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879365f7-1879-42bf-b3df-864cff303fc9 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:18.946149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:19.065790 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-879365f7-1879-42bf-b3df-864cff303fc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24e4590a76d84a40822d78ca86028d06'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:19.074065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-879365f7-1879-42bf-b3df-864cff303fc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24e4590a76d84a40822d78ca86028d06'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:19.136337 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-879365f7-1879-42bf-b3df-864cff303fc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24e4590a76d84a40822d78ca86028d06'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:19.156353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-879365f7-1879-42bf-b3df-864cff303fc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24e4590a76d84a40822d78ca86028d06'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:19.157199 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-879365f7-1879-42bf-b3df-864cff303fc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24e4590a76d84a40822d78ca86028d06'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:19.164024 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1270/2532] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:18 2022] DELETE /identity/v3/users/24e4590a76d84a40822d78ca86028d06 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:19.278804 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:19.280299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:19.341833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba08fcc-5f83-4ac0-ba01-ef18d2e09209 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:19.373538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:19.403761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ecb489-6499-4112-9b72-de4bb8961647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:19.404869 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecb489-6499-4112-9b72-de4bb8961647 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:19.405390 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecb489-6499-4112-9b72-de4bb8961647 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:19.405877 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecb489-6499-4112-9b72-de4bb8961647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:19.409637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f6ecb489-6499-4112-9b72-de4bb8961647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:19.537528 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ecb489-6499-4112-9b72-de4bb8961647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:19.540387 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ecb489-6499-4112-9b72-de4bb8961647 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:19.561741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1263/2533] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:19.820771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879365f7-1879-42bf-b3df-864cff303fc9 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:19.823149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-879365f7-1879-42bf-b3df-864cff303fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:19.838914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879365f7-1879-42bf-b3df-864cff303fc9 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:19.846174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab2d45-290e-45df-9b0c-eae59570383e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:19.846851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab2d45-290e-45df-9b0c-eae59570383e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:19.847134 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab2d45-290e-45df-9b0c-eae59570383e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:19.847414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab2d45-290e-45df-9b0c-eae59570383e admin admin] PATH_INFO: `/v3/projects/fdf82c0429824d86a923161df5d2004d` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:19.848786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab2d45-290e-45df-9b0c-eae59570383e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf82c0429824d86a923161df5d2004d)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:19.850766 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab2d45-290e-45df-9b0c-eae59570383e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:19.973332 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-54ab2d45-290e-45df-9b0c-eae59570383e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf82c0429824d86a923161df5d2004d', 'request_id': 'req-54ab2d45-290e-45df-9b0c-eae59570383e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:19.983453 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-54ab2d45-290e-45df-9b0c-eae59570383e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf82c0429824d86a923161df5d2004d', 'request_id': 'req-54ab2d45-290e-45df-9b0c-eae59570383e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:20.061558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-54ab2d45-290e-45df-9b0c-eae59570383e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf82c0429824d86a923161df5d2004d', 'request_id': 'req-54ab2d45-290e-45df-9b0c-eae59570383e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:20.075348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-54ab2d45-290e-45df-9b0c-eae59570383e admin admin] The token cache is being invalidate because project fdf82c0429824d86a923161df5d2004d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:20.075700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-54ab2d45-290e-45df-9b0c-eae59570383e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:20.076337 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-54ab2d45-290e-45df-9b0c-eae59570383e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:20.077976 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1271/2534] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:19 2022] DELETE /identity/v3/projects/fdf82c0429824d86a923161df5d2004d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:20.083670 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2471545-3f89-4ffa-8715-064c09a787ab None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:20.084210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2471545-3f89-4ffa-8715-064c09a787ab None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:20.084607 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2471545-3f89-4ffa-8715-064c09a787ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:20.111383 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-b2471545-3f89-4ffa-8715-064c09a787ab None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:20.113150 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b2471545-3f89-4ffa-8715-064c09a787ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:20.189686 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1264/2535] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:20.199057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b204f-54bb-42b8-ad7e-96e35ef5c8e8 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:20.199355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b204f-54bb-42b8-ad7e-96e35ef5c8e8 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:20.199670 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b204f-54bb-42b8-ad7e-96e35ef5c8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:20.222492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-f21b204f-54bb-42b8-ad7e-96e35ef5c8e8 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:20.223879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f21b204f-54bb-42b8-ad7e-96e35ef5c8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:20.236933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1272/2536] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:20.241574 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2471545-3f89-4ffa-8715-064c09a787ab None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:20.258458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2471545-3f89-4ffa-8715-064c09a787ab None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:20.263739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c9394-7ced-4ec7-bc62-74ea80c0dbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:20.264389 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c9394-7ced-4ec7-bc62-74ea80c0dbbf admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:20.264675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c9394-7ced-4ec7-bc62-74ea80c0dbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:20.264952 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c9394-7ced-4ec7-bc62-74ea80c0dbbf admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:20.265677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c9394-7ced-4ec7-bc62-74ea80c0dbbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:13:20.265989 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c9394-7ced-4ec7-bc62-74ea80c0dbbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:20.268269 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c9394-7ced-4ec7-bc62-74ea80c0dbbf admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:20.283096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1265/2537] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:13:20.288969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b204f-54bb-42b8-ad7e-96e35ef5c8e8 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:20.302077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b204f-54bb-42b8-ad7e-96e35ef5c8e8 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:20.307971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd14e2b-09c1-4711-8918-acf5ea403145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:20.309075 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd14e2b-09c1-4711-8918-acf5ea403145 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:20.309328 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd14e2b-09c1-4711-8918-acf5ea403145 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:20.309592 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd14e2b-09c1-4711-8918-acf5ea403145 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:20.310792 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd14e2b-09c1-4711-8918-acf5ea403145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:20.313013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd14e2b-09c1-4711-8918-acf5ea403145 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:20.515651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1273/2538] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:20 2022] POST /identity/v3/projects => generated 381 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:20.519761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c9394-7ced-4ec7-bc62-74ea80c0dbbf admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:20.531505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c9394-7ced-4ec7-bc62-74ea80c0dbbf admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:20.605836 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c96ce5-79ae-498e-8e13-c863ee7faf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:20.606590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c96ce5-79ae-498e-8e13-c863ee7faf41 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:20.606943 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c96ce5-79ae-498e-8e13-c863ee7faf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:20.607230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c96ce5-79ae-498e-8e13-c863ee7faf41 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:20.607959 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c96ce5-79ae-498e-8e13-c863ee7faf41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:20.609941 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c96ce5-79ae-498e-8e13-c863ee7faf41 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:20.719229 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:20.733727 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1266/2539] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:20 2022] POST /identity/v3/users => generated 291 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:20.738607 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd14e2b-09c1-4711-8918-acf5ea403145 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:20.753235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd14e2b-09c1-4711-8918-acf5ea403145 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:20.761077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce875b91-306b-4513-ade1-1138c677cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:20.762168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce875b91-306b-4513-ade1-1138c677cd4a admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:20.762663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce875b91-306b-4513-ade1-1138c677cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:20.763143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce875b91-306b-4513-ade1-1138c677cd4a admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:20.764148 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce875b91-306b-4513-ade1-1138c677cd4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:20.767135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce875b91-306b-4513-ade1-1138c677cd4a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:20.782935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1274/2540] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:20.784363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c96ce5-79ae-498e-8e13-c863ee7faf41 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:20.797811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c96ce5-79ae-498e-8e13-c863ee7faf41 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:20.804207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29238583-6c3f-491f-bbeb-ad22419c624c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:20.804414 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29238583-6c3f-491f-bbeb-ad22419c624c admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:20.804704 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29238583-6c3f-491f-bbeb-ad22419c624c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:20.805205 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29238583-6c3f-491f-bbeb-ad22419c624c admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:20.805777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29238583-6c3f-491f-bbeb-ad22419c624c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:20.808085 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29238583-6c3f-491f-bbeb-ad22419c624c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:20.821019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1267/2541] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:20.825755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce875b91-306b-4513-ade1-1138c677cd4a admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:20.836985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce875b91-306b-4513-ade1-1138c677cd4a admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:20.842714 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1812d-3544-498a-a637-14008e6d3e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:20.843407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1812d-3544-498a-a637-14008e6d3e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:20.843712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1812d-3544-498a-a637-14008e6d3e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:20.843993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1812d-3544-498a-a637-14008e6d3e6b admin admin] PATH_INFO: `/v3/projects/2b43793e3ee440899b338c5deee2a7d5/users/890929a1d0164951979a1618109bf790/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:20.872317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1812d-3544-498a-a637-14008e6d3e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b43793e3ee440899b338c5deee2a7d5, user_id=890929a1d0164951979a1618109bf790, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:20.877394 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1812d-3544-498a-a637-14008e6d3e6b admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:20.984780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1275/2542] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:20 2022] PUT /identity/v3/projects/2b43793e3ee440899b338c5deee2a7d5/users/890929a1d0164951979a1618109bf790/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:20.999508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457a7aa-335a-48c5-8a98-a535a464ea12 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:20.999914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457a7aa-335a-48c5-8a98-a535a464ea12 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:21.000184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457a7aa-335a-48c5-8a98-a535a464ea12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:21.041370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-e457a7aa-335a-48c5-8a98-a535a464ea12 None None] MFA Rules not processed for user `890929a1d0164951979a1618109bf790`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:21.043811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e457a7aa-335a-48c5-8a98-a535a464ea12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:21.140363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1268/2543] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:21.146445 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1812d-3544-498a-a637-14008e6d3e6b admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:21.165225 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=80,keystone:DELETE=16,keystone:INSERT=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:21.230620 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1812d-3544-498a-a637-14008e6d3e6b admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:21.237008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f0d0b-2095-4117-a9c2-d08ef56f8bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:21.238234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f0d0b-2095-4117-a9c2-d08ef56f8bbb admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:21.239034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f0d0b-2095-4117-a9c2-d08ef56f8bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:21.239786 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f0d0b-2095-4117-a9c2-d08ef56f8bbb admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:21.241093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f0d0b-2095-4117-a9c2-d08ef56f8bbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:21.245548 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f0d0b-2095-4117-a9c2-d08ef56f8bbb admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:21.412377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1276/2544] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:21 2022] POST /identity/v3/projects => generated 381 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:21.416937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457a7aa-335a-48c5-8a98-a535a464ea12 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:21.429119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457a7aa-335a-48c5-8a98-a535a464ea12 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:21.499764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01345793-26a7-4b74-83c8-ef6b9a8351d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:21.500427 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01345793-26a7-4b74-83c8-ef6b9a8351d5 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:21.500709 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01345793-26a7-4b74-83c8-ef6b9a8351d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:21.500983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01345793-26a7-4b74-83c8-ef6b9a8351d5 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:21.501669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01345793-26a7-4b74-83c8-ef6b9a8351d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:21.504161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01345793-26a7-4b74-83c8-ef6b9a8351d5 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:21.633411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1269/2545] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:21 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:13:21.637827 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f0d0b-2095-4117-a9c2-d08ef56f8bbb admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:21.655358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f0d0b-2095-4117-a9c2-d08ef56f8bbb admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:21.660923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9ed79-ffd2-4e72-863d-d751659f055d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:21.661568 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ed79-ffd2-4e72-863d-d751659f055d admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:21.661901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ed79-ffd2-4e72-863d-d751659f055d admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:21.662183 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ed79-ffd2-4e72-863d-d751659f055d admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:21.662994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9ed79-ffd2-4e72-863d-d751659f055d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:21.664657 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9ed79-ffd2-4e72-863d-d751659f055d admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:21.684256 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1277/2546] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:21.685519 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01345793-26a7-4b74-83c8-ef6b9a8351d5 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:21.698294 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01345793-26a7-4b74-83c8-ef6b9a8351d5 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:21.707178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75071d45-d32d-48f0-ad55-6b415ac48c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:21.708259 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75071d45-d32d-48f0-ad55-6b415ac48c18 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:21.708748 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75071d45-d32d-48f0-ad55-6b415ac48c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:21.709070 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75071d45-d32d-48f0-ad55-6b415ac48c18 admin admin] PATH_INFO: `/v3/projects/ce0d4ee790ac41f1ac65825b5ae81648/users/3033dea2a56342fa95f5d3e58edc6ed4/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:21.728436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75071d45-d32d-48f0-ad55-6b415ac48c18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce0d4ee790ac41f1ac65825b5ae81648, user_id=3033dea2a56342fa95f5d3e58edc6ed4, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:21.731497 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75071d45-d32d-48f0-ad55-6b415ac48c18 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:21.806760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1270/2547] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:21 2022] PUT /identity/v3/projects/ce0d4ee790ac41f1ac65825b5ae81648/users/3033dea2a56342fa95f5d3e58edc6ed4/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:21.812473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9ed79-ffd2-4e72-863d-d751659f055d admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:21.824400 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9ed79-ffd2-4e72-863d-d751659f055d admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:21.896010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169eec6-5016-4808-926a-61e505de2556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:21.896010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169eec6-5016-4808-926a-61e505de2556 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:21.896010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169eec6-5016-4808-926a-61e505de2556 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:21.896010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169eec6-5016-4808-926a-61e505de2556 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:21.896550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169eec6-5016-4808-926a-61e505de2556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:21.896550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169eec6-5016-4808-926a-61e505de2556 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:21.908901 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1278/2548] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:21.913606 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75071d45-d32d-48f0-ad55-6b415ac48c18 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:21.926931 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75071d45-d32d-48f0-ad55-6b415ac48c18 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:21.934550 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecbaef-a92d-4bb3-86f1-c61e4d4194cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:21.936201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecbaef-a92d-4bb3-86f1-c61e4d4194cf admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:21.936650 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecbaef-a92d-4bb3-86f1-c61e4d4194cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:21.941154 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecbaef-a92d-4bb3-86f1-c61e4d4194cf admin admin] PATH_INFO: `/v3/projects/ce0d4ee790ac41f1ac65825b5ae81648/users/3033dea2a56342fa95f5d3e58edc6ed4/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:21.941154 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecbaef-a92d-4bb3-86f1-c61e4d4194cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce0d4ee790ac41f1ac65825b5ae81648, user_id=3033dea2a56342fa95f5d3e58edc6ed4, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:21.944221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecbaef-a92d-4bb3-86f1-c61e4d4194cf admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:21.966926 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1271/2549] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:21 2022] PUT /identity/v3/projects/ce0d4ee790ac41f1ac65825b5ae81648/users/3033dea2a56342fa95f5d3e58edc6ed4/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:21.983714 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be1bf6-e13c-4b61-aefe-34d28680a96a None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:21.984095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be1bf6-e13c-4b61-aefe-34d28680a96a None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:21.998889 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be1bf6-e13c-4b61-aefe-34d28680a96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:22.036364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-41be1bf6-e13c-4b61-aefe-34d28680a96a None None] MFA Rules not processed for user `3033dea2a56342fa95f5d3e58edc6ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:22.038535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-41be1bf6-e13c-4b61-aefe-34d28680a96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:22.208074 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1279/2550] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:22.221064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecbaef-a92d-4bb3-86f1-c61e4d4194cf admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:22.223419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e1ecbaef-a92d-4bb3-86f1-c61e4d4194cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:22.298289 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:22.333560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecbaef-a92d-4bb3-86f1-c61e4d4194cf admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:22.338937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93842a5-a9cc-4b16-99c3-bd644babb189 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:22.339843 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93842a5-a9cc-4b16-99c3-bd644babb189 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:22.340374 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93842a5-a9cc-4b16-99c3-bd644babb189 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:22.340771 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93842a5-a9cc-4b16-99c3-bd644babb189 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:22.352313 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93842a5-a9cc-4b16-99c3-bd644babb189 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:22.353869 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93842a5-a9cc-4b16-99c3-bd644babb189 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:22.365377 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1272/2551] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:22.487045 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be1bf6-e13c-4b61-aefe-34d28680a96a None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:22.498429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be1bf6-e13c-4b61-aefe-34d28680a96a None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:22.502210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1bdc9-97fb-40f2-90d7-a57514097a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:22.502823 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1bdc9-97fb-40f2-90d7-a57514097a82 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:22.503099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1bdc9-97fb-40f2-90d7-a57514097a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:22.503376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1bdc9-97fb-40f2-90d7-a57514097a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:22.508235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:22.579128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1bdc9-97fb-40f2-90d7-a57514097a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:22.580651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1bdc9-97fb-40f2-90d7-a57514097a82 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:22.591708 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1280/2552] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:24.021719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93842a5-a9cc-4b16-99c3-bd644babb189 service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:24.023371 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b93842a5-a9cc-4b16-99c3-bd644babb189 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:24.093552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93842a5-a9cc-4b16-99c3-bd644babb189 service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:24.098176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5ce12-0cf5-452f-bf54-179ee8065eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:24.098814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5ce12-0cf5-452f-bf54-179ee8065eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:24.099096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5ce12-0cf5-452f-bf54-179ee8065eda admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:24.099372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5ce12-0cf5-452f-bf54-179ee8065eda admin admin] PATH_INFO: `/v3/users/d2738e8ef94a48bb9a0d6b1177598112` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:24.100582 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5ce12-0cf5-452f-bf54-179ee8065eda admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2738e8ef94a48bb9a0d6b1177598112)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:24.102489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5ce12-0cf5-452f-bf54-179ee8065eda admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:24.328284 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-26c5ce12-0cf5-452f-bf54-179ee8065eda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2738e8ef94a48bb9a0d6b1177598112'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:24.328967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-26c5ce12-0cf5-452f-bf54-179ee8065eda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2738e8ef94a48bb9a0d6b1177598112'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:24.329634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-26c5ce12-0cf5-452f-bf54-179ee8065eda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2738e8ef94a48bb9a0d6b1177598112'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:24.337669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-26c5ce12-0cf5-452f-bf54-179ee8065eda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2738e8ef94a48bb9a0d6b1177598112'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:24.349619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-26c5ce12-0cf5-452f-bf54-179ee8065eda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2738e8ef94a48bb9a0d6b1177598112'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:24.437151 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1273/2553] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:24 2022] DELETE /identity/v3/users/d2738e8ef94a48bb9a0d6b1177598112 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:24.800336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1bdc9-97fb-40f2-90d7-a57514097a82 service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:24.814287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1bdc9-97fb-40f2-90d7-a57514097a82 service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:24.878464 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd76216-13c6-4f22-b899-f2e82b46b6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:24.879102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd76216-13c6-4f22-b899-f2e82b46b6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:24.879414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd76216-13c6-4f22-b899-f2e82b46b6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:24.879716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd76216-13c6-4f22-b899-f2e82b46b6c0 admin admin] PATH_INFO: `/v3/projects/242289f1dedc45ccbdd6aa8e19d4e414` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:24.880872 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd76216-13c6-4f22-b899-f2e82b46b6c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=242289f1dedc45ccbdd6aa8e19d4e414)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:24.882782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd76216-13c6-4f22-b899-f2e82b46b6c0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:25.035047 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5dd76216-13c6-4f22-b899-f2e82b46b6c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '242289f1dedc45ccbdd6aa8e19d4e414', 'request_id': 'req-5dd76216-13c6-4f22-b899-f2e82b46b6c0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:25.042555 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5dd76216-13c6-4f22-b899-f2e82b46b6c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '242289f1dedc45ccbdd6aa8e19d4e414', 'request_id': 'req-5dd76216-13c6-4f22-b899-f2e82b46b6c0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:25.129804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5dd76216-13c6-4f22-b899-f2e82b46b6c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '242289f1dedc45ccbdd6aa8e19d4e414', 'request_id': 'req-5dd76216-13c6-4f22-b899-f2e82b46b6c0'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:25.142039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5dd76216-13c6-4f22-b899-f2e82b46b6c0 admin admin] The token cache is being invalidate because project 242289f1dedc45ccbdd6aa8e19d4e414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:25.142325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5dd76216-13c6-4f22-b899-f2e82b46b6c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:25.142987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5dd76216-13c6-4f22-b899-f2e82b46b6c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:25.144573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1281/2554] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:24 2022] DELETE /identity/v3/projects/242289f1dedc45ccbdd6aa8e19d4e414 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:25.150277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442fa11-a0c7-44ce-a14c-6b4136600809 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:25.150794 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442fa11-a0c7-44ce-a14c-6b4136600809 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:25.150891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442fa11-a0c7-44ce-a14c-6b4136600809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:25.171953 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-d442fa11-a0c7-44ce-a14c-6b4136600809 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:25.173184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d442fa11-a0c7-44ce-a14c-6b4136600809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:25.222677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:25.238411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1274/2555] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:25.246724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd705b-eaef-4eaa-90f1-0596f17249ac None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:25.247014 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd705b-eaef-4eaa-90f1-0596f17249ac None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:25.247432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd705b-eaef-4eaa-90f1-0596f17249ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:25.267757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-47bd705b-eaef-4eaa-90f1-0596f17249ac None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:25.268975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-47bd705b-eaef-4eaa-90f1-0596f17249ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:25.277759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1282/2556] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:25.282338 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442fa11-a0c7-44ce-a14c-6b4136600809 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:25.294051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442fa11-a0c7-44ce-a14c-6b4136600809 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:25.299187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c2909-f325-402c-b17b-e934cc5fb66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:25.299902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c2909-f325-402c-b17b-e934cc5fb66c admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:25.300185 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c2909-f325-402c-b17b-e934cc5fb66c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:25.301386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c2909-f325-402c-b17b-e934cc5fb66c admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:25.301386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c2909-f325-402c-b17b-e934cc5fb66c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:13:25.301641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c2909-f325-402c-b17b-e934cc5fb66c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:25.303056 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c2909-f325-402c-b17b-e934cc5fb66c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:25.315831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1275/2557] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:13:25.320415 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd705b-eaef-4eaa-90f1-0596f17249ac None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:25.331019 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd705b-eaef-4eaa-90f1-0596f17249ac None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:25.335721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ddebd-970a-4afd-bb84-027888201cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:25.336441 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ddebd-970a-4afd-bb84-027888201cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:25.336735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ddebd-970a-4afd-bb84-027888201cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:25.347447 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ddebd-970a-4afd-bb84-027888201cb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:25.347447 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ddebd-970a-4afd-bb84-027888201cb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:25.347447 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ddebd-970a-4afd-bb84-027888201cb1 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:25.503637 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1283/2558] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:25 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:13:25.507074 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c2909-f325-402c-b17b-e934cc5fb66c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:25.517682 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c2909-f325-402c-b17b-e934cc5fb66c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:25.575549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bc071-246f-40e7-ab63-0aacbc06cdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:25.576167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bc071-246f-40e7-ab63-0aacbc06cdf3 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:25.576474 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bc071-246f-40e7-ab63-0aacbc06cdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:25.576756 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bc071-246f-40e7-ab63-0aacbc06cdf3 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:25.577425 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bc071-246f-40e7-ab63-0aacbc06cdf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:25.579378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bc071-246f-40e7-ab63-0aacbc06cdf3 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:25.667582 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1276/2559] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:25 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:25.671951 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ddebd-970a-4afd-bb84-027888201cb1 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:25.682408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ddebd-970a-4afd-bb84-027888201cb1 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:25.687430 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645b635-3381-4058-b7e2-b3d3afa66be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:25.688143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645b635-3381-4058-b7e2-b3d3afa66be5 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:25.688438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645b635-3381-4058-b7e2-b3d3afa66be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:25.688719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645b635-3381-4058-b7e2-b3d3afa66be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:25.689456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645b635-3381-4058-b7e2-b3d3afa66be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:25.691128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645b635-3381-4058-b7e2-b3d3afa66be5 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:25.692661 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:25.703007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1284/2560] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:25.707357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bc071-246f-40e7-ab63-0aacbc06cdf3 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:25.720060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bc071-246f-40e7-ab63-0aacbc06cdf3 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:25.724979 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c30c4-eec5-4d7f-8108-d5c308bf1937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:25.725595 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c30c4-eec5-4d7f-8108-d5c308bf1937 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:25.725882 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c30c4-eec5-4d7f-8108-d5c308bf1937 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:25.726157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c30c4-eec5-4d7f-8108-d5c308bf1937 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:25.726873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c30c4-eec5-4d7f-8108-d5c308bf1937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:25.728734 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c30c4-eec5-4d7f-8108-d5c308bf1937 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:25.740473 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1277/2561] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:25.746104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645b635-3381-4058-b7e2-b3d3afa66be5 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:25.760143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645b635-3381-4058-b7e2-b3d3afa66be5 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:25.763280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592f8e2-6648-413a-9700-d62cfa174eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:25.763999 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592f8e2-6648-413a-9700-d62cfa174eb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:25.764274 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592f8e2-6648-413a-9700-d62cfa174eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:25.764543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592f8e2-6648-413a-9700-d62cfa174eb6 admin admin] PATH_INFO: `/v3/projects/33ae25a451b84fc0bc93ee94b87e098e/users/0327648e6ad549f5b4aa8b5839e3f9af/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:25.780739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592f8e2-6648-413a-9700-d62cfa174eb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33ae25a451b84fc0bc93ee94b87e098e, user_id=0327648e6ad549f5b4aa8b5839e3f9af, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:25.783390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592f8e2-6648-413a-9700-d62cfa174eb6 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:25.878838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1285/2562] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:25 2022] PUT /identity/v3/projects/33ae25a451b84fc0bc93ee94b87e098e/users/0327648e6ad549f5b4aa8b5839e3f9af/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:25.891807 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61f485-e9f9-4033-a449-89982297bca1 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:25.892072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61f485-e9f9-4033-a449-89982297bca1 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:25.897231 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61f485-e9f9-4033-a449-89982297bca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:25.923380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592f8e2-6648-413a-9700-d62cfa174eb6 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:25.925659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ab61f485-e9f9-4033-a449-89982297bca1 None None] MFA Rules not processed for user `0327648e6ad549f5b4aa8b5839e3f9af`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:25.926880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ab61f485-e9f9-4033-a449-89982297bca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:25.932761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a592f8e2-6648-413a-9700-d62cfa174eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:26.004040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1278/2563] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:26.007235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592f8e2-6648-413a-9700-d62cfa174eb6 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:26.015145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f722e9f-9835-4c5a-af5e-ceaf322edbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:26.015985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f722e9f-9835-4c5a-af5e-ceaf322edbaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:26.016128 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61f485-e9f9-4033-a449-89982297bca1 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:26.016266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f722e9f-9835-4c5a-af5e-ceaf322edbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:26.016536 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f722e9f-9835-4c5a-af5e-ceaf322edbaa admin admin] PATH_INFO: `/v3/users/890929a1d0164951979a1618109bf790` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:26.017651 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ab61f485-e9f9-4033-a449-89982297bca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:26.017812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f722e9f-9835-4c5a-af5e-ceaf322edbaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=890929a1d0164951979a1618109bf790)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:26.019916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f722e9f-9835-4c5a-af5e-ceaf322edbaa admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:26.077759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:26.128375 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61f485-e9f9-4033-a449-89982297bca1 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:26.134623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3933131-c831-4930-b660-19bc6f25c134 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:26.135267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3933131-c831-4930-b660-19bc6f25c134 service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:26.135560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3933131-c831-4930-b660-19bc6f25c134 service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:26.135838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3933131-c831-4930-b660-19bc6f25c134 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:26.144483 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3933131-c831-4930-b660-19bc6f25c134 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:26.146046 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3933131-c831-4930-b660-19bc6f25c134 service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:26.157982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1279/2564] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:13:26.254287 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5f722e9f-9835-4c5a-af5e-ceaf322edbaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '890929a1d0164951979a1618109bf790'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:26.261601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5f722e9f-9835-4c5a-af5e-ceaf322edbaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '890929a1d0164951979a1618109bf790'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:26.358376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5f722e9f-9835-4c5a-af5e-ceaf322edbaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '890929a1d0164951979a1618109bf790'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:26.371801 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5f722e9f-9835-4c5a-af5e-ceaf322edbaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '890929a1d0164951979a1618109bf790'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:26.372503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-5f722e9f-9835-4c5a-af5e-ceaf322edbaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '890929a1d0164951979a1618109bf790'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:26.378510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1286/2565] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:25 2022] DELETE /identity/v3/users/890929a1d0164951979a1618109bf790 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:26.383217 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ba9c2-8638-41e6-8b79-af72773841a9 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:26.383545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ba9c2-8638-41e6-8b79-af72773841a9 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:26.383829 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ba9c2-8638-41e6-8b79-af72773841a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:26.404146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-d87ba9c2-8638-41e6-8b79-af72773841a9 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:26.410365 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d87ba9c2-8638-41e6-8b79-af72773841a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:26.466298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1280/2566] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:26.476597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f722e9f-9835-4c5a-af5e-ceaf322edbaa admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:26.478254 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-5f722e9f-9835-4c5a-af5e-ceaf322edbaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:26.539566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f722e9f-9835-4c5a-af5e-ceaf322edbaa admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:26.543403 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5e3c5-631b-48e8-9e63-45f59601d771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:26.544054 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5e3c5-631b-48e8-9e63-45f59601d771 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:26.544330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5e3c5-631b-48e8-9e63-45f59601d771 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:26.544597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5e3c5-631b-48e8-9e63-45f59601d771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:26.550744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:26.554520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5e3c5-631b-48e8-9e63-45f59601d771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:26.556578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5e3c5-631b-48e8-9e63-45f59601d771 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:26.566462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1287/2567] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:13:27.051785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ba9c2-8638-41e6-8b79-af72773841a9 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:27.053338 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d87ba9c2-8638-41e6-8b79-af72773841a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:27.067413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ba9c2-8638-41e6-8b79-af72773841a9 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:27.072627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:27.073458 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:27.073742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:27.074020 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] PATH_INFO: `/v3/projects/2b43793e3ee440899b338c5deee2a7d5` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:27.075252 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b43793e3ee440899b338c5deee2a7d5)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:27.077223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:27.265771 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b43793e3ee440899b338c5deee2a7d5', 'request_id': 'req-8396221b-64d2-42d0-a80d-6bdd51135c43'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:27.272369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b43793e3ee440899b338c5deee2a7d5', 'request_id': 'req-8396221b-64d2-42d0-a80d-6bdd51135c43'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:27.278890 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b43793e3ee440899b338c5deee2a7d5', 'request_id': 'req-8396221b-64d2-42d0-a80d-6bdd51135c43'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:27.398845 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] The token cache is being invalidate because project 2b43793e3ee440899b338c5deee2a7d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:27.399078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:27.400318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:27.401894 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1281/2568] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:27 2022] DELETE /identity/v3/projects/2b43793e3ee440899b338c5deee2a7d5 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:27.406268 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5e3c5-631b-48e8-9e63-45f59601d771 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:27.417170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5e3c5-631b-48e8-9e63-45f59601d771 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:27.474558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9657629-a2d5-483b-a8e1-d4d7eccf1a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:27.475190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9657629-a2d5-483b-a8e1-d4d7eccf1a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:27.475491 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9657629-a2d5-483b-a8e1-d4d7eccf1a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:27.475771 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9657629-a2d5-483b-a8e1-d4d7eccf1a39 admin admin] PATH_INFO: `/v3/users/3033dea2a56342fa95f5d3e58edc6ed4` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:27.476958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9657629-a2d5-483b-a8e1-d4d7eccf1a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3033dea2a56342fa95f5d3e58edc6ed4)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:27.479729 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9657629-a2d5-483b-a8e1-d4d7eccf1a39 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:27.716634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a9657629-a2d5-483b-a8e1-d4d7eccf1a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3033dea2a56342fa95f5d3e58edc6ed4'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:27.724181 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a9657629-a2d5-483b-a8e1-d4d7eccf1a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3033dea2a56342fa95f5d3e58edc6ed4'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:27.786727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a9657629-a2d5-483b-a8e1-d4d7eccf1a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3033dea2a56342fa95f5d3e58edc6ed4'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:27.800643 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a9657629-a2d5-483b-a8e1-d4d7eccf1a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3033dea2a56342fa95f5d3e58edc6ed4'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:27.801349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a9657629-a2d5-483b-a8e1-d4d7eccf1a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3033dea2a56342fa95f5d3e58edc6ed4'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:27.807549 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1288/2569] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:27 2022] DELETE /identity/v3/users/3033dea2a56342fa95f5d3e58edc6ed4 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:28.277659 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:28.279298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:28.296374 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396221b-64d2-42d0-a80d-6bdd51135c43 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:28.362322 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:28.393864 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540b19a-772d-4630-ab50-7dd01a86430a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:28.397271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540b19a-772d-4630-ab50-7dd01a86430a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:28.397271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540b19a-772d-4630-ab50-7dd01a86430a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:28.397271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540b19a-772d-4630-ab50-7dd01a86430a admin admin] PATH_INFO: `/v3/projects/ce0d4ee790ac41f1ac65825b5ae81648` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:28.397271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540b19a-772d-4630-ab50-7dd01a86430a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce0d4ee790ac41f1ac65825b5ae81648)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:28.401209 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540b19a-772d-4630-ab50-7dd01a86430a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:28.538174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7540b19a-772d-4630-ab50-7dd01a86430a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce0d4ee790ac41f1ac65825b5ae81648', 'request_id': 'req-7540b19a-772d-4630-ab50-7dd01a86430a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:28.545404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7540b19a-772d-4630-ab50-7dd01a86430a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce0d4ee790ac41f1ac65825b5ae81648', 'request_id': 'req-7540b19a-772d-4630-ab50-7dd01a86430a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:28.551692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7540b19a-772d-4630-ab50-7dd01a86430a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce0d4ee790ac41f1ac65825b5ae81648', 'request_id': 'req-7540b19a-772d-4630-ab50-7dd01a86430a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:28.650885 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7540b19a-772d-4630-ab50-7dd01a86430a admin admin] The token cache is being invalidate because project ce0d4ee790ac41f1ac65825b5ae81648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:28.651239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7540b19a-772d-4630-ab50-7dd01a86430a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:28.652154 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7540b19a-772d-4630-ab50-7dd01a86430a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:28.653961 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1282/2570] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:28 2022] DELETE /identity/v3/projects/ce0d4ee790ac41f1ac65825b5ae81648 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:28.659759 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1607dd-1197-4d80-8669-09a8955ef22e None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:28.660626 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1607dd-1197-4d80-8669-09a8955ef22e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:28.660626 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1607dd-1197-4d80-8669-09a8955ef22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:28.685325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-2a1607dd-1197-4d80-8669-09a8955ef22e None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:28.687117 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2a1607dd-1197-4d80-8669-09a8955ef22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:28.769171 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1289/2571] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:28.777706 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e6064-942c-4f75-86b5-5d77018bc0b3 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:28.777991 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e6064-942c-4f75-86b5-5d77018bc0b3 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:28.781806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e6064-942c-4f75-86b5-5d77018bc0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:28.798721 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-aa9e6064-942c-4f75-86b5-5d77018bc0b3 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:28.800021 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-aa9e6064-942c-4f75-86b5-5d77018bc0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:28.809766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1283/2572] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:28.814486 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1607dd-1197-4d80-8669-09a8955ef22e None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:28.827096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1607dd-1197-4d80-8669-09a8955ef22e None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:28.832835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d7b4b-0e93-475c-822e-a83243b34453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:28.833769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d7b4b-0e93-475c-822e-a83243b34453 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:28.834064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d7b4b-0e93-475c-822e-a83243b34453 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:28.834437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d7b4b-0e93-475c-822e-a83243b34453 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:28.835404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d7b4b-0e93-475c-822e-a83243b34453 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:13:28.835841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d7b4b-0e93-475c-822e-a83243b34453 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:28.838056 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d7b4b-0e93-475c-822e-a83243b34453 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:28.848064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:28.852684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1290/2573] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:13:28.856011 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e6064-942c-4f75-86b5-5d77018bc0b3 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:28.866422 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e6064-942c-4f75-86b5-5d77018bc0b3 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:28.872123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd466936-0efe-4974-ae48-d6a8e19c4e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:28.872753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd466936-0efe-4974-ae48-d6a8e19c4e79 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:28.873076 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd466936-0efe-4974-ae48-d6a8e19c4e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:28.873326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd466936-0efe-4974-ae48-d6a8e19c4e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:28.874003 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd466936-0efe-4974-ae48-d6a8e19c4e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:28.875942 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd466936-0efe-4974-ae48-d6a8e19c4e79 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:29.018019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1284/2574] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:28 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:29.022575 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d7b4b-0e93-475c-822e-a83243b34453 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:29.033013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d7b4b-0e93-475c-822e-a83243b34453 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:29.095722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c386d2b-a883-41b9-b984-550e89e5c55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:29.096305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c386d2b-a883-41b9-b984-550e89e5c55f admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:29.096582 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c386d2b-a883-41b9-b984-550e89e5c55f admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:29.096859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c386d2b-a883-41b9-b984-550e89e5c55f admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:29.097672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c386d2b-a883-41b9-b984-550e89e5c55f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:29.099732 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c386d2b-a883-41b9-b984-550e89e5c55f admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:29.210075 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1291/2575] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:29 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:13:29.214752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd466936-0efe-4974-ae48-d6a8e19c4e79 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:29.225724 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd466936-0efe-4974-ae48-d6a8e19c4e79 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:29.231009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf46211-d2e6-42c3-b3ef-770ce88199ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:29.231682 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf46211-d2e6-42c3-b3ef-770ce88199ac admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:29.231967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf46211-d2e6-42c3-b3ef-770ce88199ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:29.232244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf46211-d2e6-42c3-b3ef-770ce88199ac admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:29.232949 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf46211-d2e6-42c3-b3ef-770ce88199ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:29.234566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf46211-d2e6-42c3-b3ef-770ce88199ac admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:29.246902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1285/2576] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:29.252145 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c386d2b-a883-41b9-b984-550e89e5c55f admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:29.267514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c386d2b-a883-41b9-b984-550e89e5c55f admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:29.276494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0c336-e7f2-4b21-8c8a-5fed7678c5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:29.277522 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0c336-e7f2-4b21-8c8a-5fed7678c5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:29.278032 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0c336-e7f2-4b21-8c8a-5fed7678c5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:29.278496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0c336-e7f2-4b21-8c8a-5fed7678c5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:29.279641 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0c336-e7f2-4b21-8c8a-5fed7678c5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:29.282396 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea0c336-e7f2-4b21-8c8a-5fed7678c5a4 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:29.298815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1292/2577] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:29.303158 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf46211-d2e6-42c3-b3ef-770ce88199ac admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:29.314853 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf46211-d2e6-42c3-b3ef-770ce88199ac admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:29.319711 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc81aa-2c70-413e-a2ff-abe55bbcfebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:29.320394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc81aa-2c70-413e-a2ff-abe55bbcfebc admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:29.320677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc81aa-2c70-413e-a2ff-abe55bbcfebc admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:29.320951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc81aa-2c70-413e-a2ff-abe55bbcfebc admin admin] PATH_INFO: `/v3/projects/912b783c262149598bbc2b398aec7ab4/users/3009e4514812421a87e7e604d2326556/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:29.337329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc81aa-2c70-413e-a2ff-abe55bbcfebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=912b783c262149598bbc2b398aec7ab4, user_id=3009e4514812421a87e7e604d2326556, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:29.340611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc81aa-2c70-413e-a2ff-abe55bbcfebc admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:29.440128 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1286/2578] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:29 2022] PUT /identity/v3/projects/912b783c262149598bbc2b398aec7ab4/users/3009e4514812421a87e7e604d2326556/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:29.449930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a3d89-f6ed-4eea-b0b8-404d24baf1af None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:29.450429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a3d89-f6ed-4eea-b0b8-404d24baf1af None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:29.451516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a3d89-f6ed-4eea-b0b8-404d24baf1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:29.491337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc81aa-2c70-413e-a2ff-abe55bbcfebc admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:29.493986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0afc81aa-2c70-413e-a2ff-abe55bbcfebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:29.495508 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-023a3d89-f6ed-4eea-b0b8-404d24baf1af None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:29.498246 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-023a3d89-f6ed-4eea-b0b8-404d24baf1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:29.578524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1293/2579] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:29.579729 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc81aa-2c70-413e-a2ff-abe55bbcfebc admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:29.585568 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6124319-91b7-43a7-8007-ee764f1130fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:29.586244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6124319-91b7-43a7-8007-ee764f1130fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:29.586525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6124319-91b7-43a7-8007-ee764f1130fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:29.586794 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6124319-91b7-43a7-8007-ee764f1130fc admin admin] PATH_INFO: `/v3/users/0327648e6ad549f5b4aa8b5839e3f9af` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:29.588653 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6124319-91b7-43a7-8007-ee764f1130fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0327648e6ad549f5b4aa8b5839e3f9af)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:29.590555 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6124319-91b7-43a7-8007-ee764f1130fc admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:29.592206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a3d89-f6ed-4eea-b0b8-404d24baf1af None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:29.593788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-023a3d89-f6ed-4eea-b0b8-404d24baf1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:29.607577 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:29.657780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a3d89-f6ed-4eea-b0b8-404d24baf1af None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:29.661312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd35c8f-403f-442a-8c61-0466e8520f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:29.661921 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd35c8f-403f-442a-8c61-0466e8520f2f service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:29.662203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd35c8f-403f-442a-8c61-0466e8520f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:29.662473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd35c8f-403f-442a-8c61-0466e8520f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:29.673027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd35c8f-403f-442a-8c61-0466e8520f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:29.674456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd35c8f-403f-442a-8c61-0466e8520f2f service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:29.684376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1294/2580] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:29.817641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b6124319-91b7-43a7-8007-ee764f1130fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0327648e6ad549f5b4aa8b5839e3f9af'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:29.818698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b6124319-91b7-43a7-8007-ee764f1130fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0327648e6ad549f5b4aa8b5839e3f9af'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:29.819921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b6124319-91b7-43a7-8007-ee764f1130fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0327648e6ad549f5b4aa8b5839e3f9af'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:29.832216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b6124319-91b7-43a7-8007-ee764f1130fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0327648e6ad549f5b4aa8b5839e3f9af'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:29.849348 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b6124319-91b7-43a7-8007-ee764f1130fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0327648e6ad549f5b4aa8b5839e3f9af'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:29.952948 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1287/2581] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:29 2022] DELETE /identity/v3/users/0327648e6ad549f5b4aa8b5839e3f9af => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:29.957454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ab731-2ed3-4493-80d8-65e254b807a8 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:29.957832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ab731-2ed3-4493-80d8-65e254b807a8 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:29.958112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ab731-2ed3-4493-80d8-65e254b807a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:29.978434 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-6f2ab731-2ed3-4493-80d8-65e254b807a8 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:29.979693 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6f2ab731-2ed3-4493-80d8-65e254b807a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:29.998964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:30.046754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1295/2582] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:30.056955 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6124319-91b7-43a7-8007-ee764f1130fc admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:30.058636 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b6124319-91b7-43a7-8007-ee764f1130fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:30.141690 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6124319-91b7-43a7-8007-ee764f1130fc admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:30.155173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfddd48-3c0d-4468-9572-fd335aaa64aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:30.155173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfddd48-3c0d-4468-9572-fd335aaa64aa service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:30.155173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfddd48-3c0d-4468-9572-fd335aaa64aa service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:30.155173 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfddd48-3c0d-4468-9572-fd335aaa64aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:30.160101 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfddd48-3c0d-4468-9572-fd335aaa64aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:30.161718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfddd48-3c0d-4468-9572-fd335aaa64aa service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:30.174099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1288/2583] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:30.645524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ab731-2ed3-4493-80d8-65e254b807a8 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:30.648129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6f2ab731-2ed3-4493-80d8-65e254b807a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:30.672013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ab731-2ed3-4493-80d8-65e254b807a8 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:30.677573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:30.678297 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:30.683212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:30.683212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da admin admin] PATH_INFO: `/v3/projects/33ae25a451b84fc0bc93ee94b87e098e` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:30.683212 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=33ae25a451b84fc0bc93ee94b87e098e)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:30.684429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:30.814836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33ae25a451b84fc0bc93ee94b87e098e', 'request_id': 'req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:30.823592 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33ae25a451b84fc0bc93ee94b87e098e', 'request_id': 'req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:30.935769 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33ae25a451b84fc0bc93ee94b87e098e', 'request_id': 'req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:30.957719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da admin admin] The token cache is being invalidate because project 33ae25a451b84fc0bc93ee94b87e098e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:30.957719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:30.957719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f22d2746-6cf4-4df9-9a28-c73a19aaf2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:30.957719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1296/2584] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:30 2022] DELETE /identity/v3/projects/33ae25a451b84fc0bc93ee94b87e098e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:30.962688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4c308-2635-4f2b-b2fd-825b501cdeaa None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:30.963324 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4c308-2635-4f2b-b2fd-825b501cdeaa None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:30.963324 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4c308-2635-4f2b-b2fd-825b501cdeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:30.990070 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-71f4c308-2635-4f2b-b2fd-825b501cdeaa None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:30.992292 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-71f4c308-2635-4f2b-b2fd-825b501cdeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:31.064770 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1289/2585] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:31.073597 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb2f11-8048-4ccd-8143-77b9fee89d55 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:31.073970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb2f11-8048-4ccd-8143-77b9fee89d55 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:31.074369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb2f11-8048-4ccd-8143-77b9fee89d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:31.096459 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-d7eb2f11-8048-4ccd-8143-77b9fee89d55 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:31.097698 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d7eb2f11-8048-4ccd-8143-77b9fee89d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:31.106470 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1297/2586] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:13:31.111918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4c308-2635-4f2b-b2fd-825b501cdeaa None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:31.123024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4c308-2635-4f2b-b2fd-825b501cdeaa None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:31.127975 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30a9ab-6cd7-4e63-ae2e-7b0c2b2a09a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:31.128604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30a9ab-6cd7-4e63-ae2e-7b0c2b2a09a8 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:31.128883 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30a9ab-6cd7-4e63-ae2e-7b0c2b2a09a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:31.129159 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30a9ab-6cd7-4e63-ae2e-7b0c2b2a09a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:31.129859 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30a9ab-6cd7-4e63-ae2e-7b0c2b2a09a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:13:31.130177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30a9ab-6cd7-4e63-ae2e-7b0c2b2a09a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:31.131806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30a9ab-6cd7-4e63-ae2e-7b0c2b2a09a8 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:31.153372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1290/2587] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:13:31.153483 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb2f11-8048-4ccd-8143-77b9fee89d55 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:31.161891 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb2f11-8048-4ccd-8143-77b9fee89d55 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:31.167221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e3635-a2a1-4301-bd63-51a673bcd63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:31.167940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e3635-a2a1-4301-bd63-51a673bcd63b admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:31.168234 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e3635-a2a1-4301-bd63-51a673bcd63b admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:31.168504 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e3635-a2a1-4301-bd63-51a673bcd63b admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:31.169185 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e3635-a2a1-4301-bd63-51a673bcd63b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:31.171566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e3635-a2a1-4301-bd63-51a673bcd63b admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:31.272645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1298/2588] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:31 2022] POST /identity/v3/projects => generated 403 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:13:31.277407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30a9ab-6cd7-4e63-ae2e-7b0c2b2a09a8 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:31.288402 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30a9ab-6cd7-4e63-ae2e-7b0c2b2a09a8 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:31.298237 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:31.314872 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaab646-d251-44b8-bdae-12051ad982d4 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:31.315179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaab646-d251-44b8-bdae-12051ad982d4 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:31.315617 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaab646-d251-44b8-bdae-12051ad982d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:31.350438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-caaab646-d251-44b8-bdae-12051ad982d4 None None] MFA Rules not processed for user `3009e4514812421a87e7e604d2326556`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:31.355893 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-caaab646-d251-44b8-bdae-12051ad982d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:31.387400 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b4933-6664-4b12-a88c-434a5d6e2ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:31.388350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b4933-6664-4b12-a88c-434a5d6e2ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:31.388647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b4933-6664-4b12-a88c-434a5d6e2ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:31.388927 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b4933-6664-4b12-a88c-434a5d6e2ef5 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:31.389623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b4933-6664-4b12-a88c-434a5d6e2ef5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:31.391819 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b4933-6664-4b12-a88c-434a5d6e2ef5 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:31.435648 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1299/2589] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:31.441069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaab646-d251-44b8-bdae-12051ad982d4 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:31.443134 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-caaab646-d251-44b8-bdae-12051ad982d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:31.459836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaab646-d251-44b8-bdae-12051ad982d4 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:31.465747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc5046-8538-40e1-b18c-4b56e8ac0d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:31.466513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc5046-8538-40e1-b18c-4b56e8ac0d57 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:31.466837 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc5046-8538-40e1-b18c-4b56e8ac0d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:31.467121 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc5046-8538-40e1-b18c-4b56e8ac0d57 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:31.467852 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc5046-8538-40e1-b18c-4b56e8ac0d57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:31.471791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc5046-8538-40e1-b18c-4b56e8ac0d57 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:31.474806 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:31.556739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1291/2590] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:31 2022] POST /identity/v3/users => generated 302 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:31.561532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b4933-6664-4b12-a88c-434a5d6e2ef5 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:31.573681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b4933-6664-4b12-a88c-434a5d6e2ef5 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:31.580379 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa458cf-8413-4fb6-879c-e1f60b06ef9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:31.581021 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa458cf-8413-4fb6-879c-e1f60b06ef9f admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:31.581303 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa458cf-8413-4fb6-879c-e1f60b06ef9f admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:31.581580 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa458cf-8413-4fb6-879c-e1f60b06ef9f admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:31.582306 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa458cf-8413-4fb6-879c-e1f60b06ef9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:31.584419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa458cf-8413-4fb6-879c-e1f60b06ef9f admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:31.601981 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1292/2591] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:31.603954 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa458cf-8413-4fb6-879c-e1f60b06ef9f admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:31.616173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1300/2592] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:31 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:13:31.621709 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa458cf-8413-4fb6-879c-e1f60b06ef9f admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:31.624748 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc5046-8538-40e1-b18c-4b56e8ac0d57 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:31.702204 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18c75d-f63a-4cb1-abfc-f629d6f6d729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:31.703117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18c75d-f63a-4cb1-abfc-f629d6f6d729 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:31.703442 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18c75d-f63a-4cb1-abfc-f629d6f6d729 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:31.703827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18c75d-f63a-4cb1-abfc-f629d6f6d729 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:31.704548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18c75d-f63a-4cb1-abfc-f629d6f6d729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:31.706176 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18c75d-f63a-4cb1-abfc-f629d6f6d729 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:31.719995 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1293/2593] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:31.721549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f715133-7d05-46cb-a317-702f50c6fd41 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:31.722497 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f715133-7d05-46cb-a317-702f50c6fd41 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:31.722785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f715133-7d05-46cb-a317-702f50c6fd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:31.725773 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc5046-8538-40e1-b18c-4b56e8ac0d57 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:31.731324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056662a-a547-4119-ac1f-7b5a180b51d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:31.732096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056662a-a547-4119-ac1f-7b5a180b51d8 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:31.732380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056662a-a547-4119-ac1f-7b5a180b51d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:31.732758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056662a-a547-4119-ac1f-7b5a180b51d8 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:31.733465 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056662a-a547-4119-ac1f-7b5a180b51d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:31.735687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056662a-a547-4119-ac1f-7b5a180b51d8 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:31.744153 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-6f715133-7d05-46cb-a317-702f50c6fd41 None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:31.745830 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-6f715133-7d05-46cb-a317-702f50c6fd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:31.805298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1294/2594] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 15:13:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:31.807322 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f715133-7d05-46cb-a317-702f50c6fd41 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:31.820063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f715133-7d05-46cb-a317-702f50c6fd41 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:31.826454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1301/2595] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:31 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:31.826454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056662a-a547-4119-ac1f-7b5a180b51d8 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:31.826729 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22925589-b4ab-45d5-bd1e-f8287f85ccfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:31.826856 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2056662a-a547-4119-ac1f-7b5a180b51d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:31.826962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22925589-b4ab-45d5-bd1e-f8287f85ccfe admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:31.826962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22925589-b4ab-45d5-bd1e-f8287f85ccfe admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:31.826962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22925589-b4ab-45d5-bd1e-f8287f85ccfe admin admin] PATH_INFO: `/v3/projects/5ada2790f9fc4118b4cefbcc7b8d1884/users/15eaadbcbda540f4a051b9d1fe7c3804/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:31.843287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22925589-b4ab-45d5-bd1e-f8287f85ccfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ada2790f9fc4118b4cefbcc7b8d1884, user_id=15eaadbcbda540f4a051b9d1fe7c3804, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:31.845958 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22925589-b4ab-45d5-bd1e-f8287f85ccfe admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:31.895634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056662a-a547-4119-ac1f-7b5a180b51d8 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:31.901263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1e81f-7fd0-4b3a-a0e4-09490a11cf84 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:31.901891 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1e81f-7fd0-4b3a-a0e4-09490a11cf84 service swift] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:31.902222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1e81f-7fd0-4b3a-a0e4-09490a11cf84 service swift] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:31.902542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1e81f-7fd0-4b3a-a0e4-09490a11cf84 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:31.911296 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1e81f-7fd0-4b3a-a0e4-09490a11cf84 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:31.912974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1e81f-7fd0-4b3a-a0e4-09490a11cf84 service swift] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:31.922555 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1302/2596] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:13:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:13:31.923408 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1e81f-7fd0-4b3a-a0e4-09490a11cf84 service swift] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:31.939584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1e81f-7fd0-4b3a-a0e4-09490a11cf84 service swift] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:31.944454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9003b9-af53-4da5-a1ff-25c854eed521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:31.945124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9003b9-af53-4da5-a1ff-25c854eed521 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:31.945396 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9003b9-af53-4da5-a1ff-25c854eed521 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:31.945666 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9003b9-af53-4da5-a1ff-25c854eed521 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:31.947015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9003b9-af53-4da5-a1ff-25c854eed521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:31.948748 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9003b9-af53-4da5-a1ff-25c854eed521 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:31.954709 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1295/2597] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:31 2022] PUT /identity/v3/projects/5ada2790f9fc4118b4cefbcc7b8d1884/users/15eaadbcbda540f4a051b9d1fe7c3804/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:31.962427 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1303/2598] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:31.964368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce9b7-2a35-46ed-8905-078809cea229 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:31.964860 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce9b7-2a35-46ed-8905-078809cea229 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:31.964983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce9b7-2a35-46ed-8905-078809cea229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:31.966311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9003b9-af53-4da5-a1ff-25c854eed521 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:31.977567 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9003b9-af53-4da5-a1ff-25c854eed521 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:31.979643 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:31.997290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-c54ce9b7-2a35-46ed-8905-078809cea229 None None] MFA Rules not processed for user `15eaadbcbda540f4a051b9d1fe7c3804`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:31.998662 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c54ce9b7-2a35-46ed-8905-078809cea229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:32.032559 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:32.046337 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e336e7a-95ae-4b16-83ba-49bda1b881c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:32.046849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e336e7a-95ae-4b16-83ba-49bda1b881c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:32.047133 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e336e7a-95ae-4b16-83ba-49bda1b881c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:32.047410 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e336e7a-95ae-4b16-83ba-49bda1b881c0 admin admin] PATH_INFO: `/v3/projects/f133d4db3602405dbd1717f77ac9d09c/users/b5a1e90bb5b940c0a8340e1fda48a12a/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:32.067388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e336e7a-95ae-4b16-83ba-49bda1b881c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f133d4db3602405dbd1717f77ac9d09c, user_id=b5a1e90bb5b940c0a8340e1fda48a12a, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:32.070178 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e336e7a-95ae-4b16-83ba-49bda1b881c0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:32.102008 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1296/2599] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:13:32.107081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ce9b7-2a35-46ed-8905-078809cea229 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:32.122151 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ce9b7-2a35-46ed-8905-078809cea229 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:32.126855 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da39dc-245c-4486-8720-b59299be7142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:32.127512 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da39dc-245c-4486-8720-b59299be7142 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:32.127806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da39dc-245c-4486-8720-b59299be7142 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:32.128254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da39dc-245c-4486-8720-b59299be7142 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:32.128943 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da39dc-245c-4486-8720-b59299be7142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:32.131079 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da39dc-245c-4486-8720-b59299be7142 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:32.144476 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1304/2600] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:31 2022] PUT /identity/v3/projects/f133d4db3602405dbd1717f77ac9d09c/users/b5a1e90bb5b940c0a8340e1fda48a12a/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:32.149025 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e336e7a-95ae-4b16-83ba-49bda1b881c0 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:32.238247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e336e7a-95ae-4b16-83ba-49bda1b881c0 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:32.245930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b57a9-bc9a-455d-8f4c-8ade321e1cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:32.245930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b57a9-bc9a-455d-8f4c-8ade321e1cfb admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:32.245930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b57a9-bc9a-455d-8f4c-8ade321e1cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:32.245930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b57a9-bc9a-455d-8f4c-8ade321e1cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:32.246452 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b57a9-bc9a-455d-8f4c-8ade321e1cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:32.247361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b57a9-bc9a-455d-8f4c-8ade321e1cfb admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:32.248087 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1297/2601] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:32 2022] POST /identity/v3/projects => generated 405 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:13:32.249495 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da39dc-245c-4486-8720-b59299be7142 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:32.266783 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1305/2602] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:32.271347 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b57a9-bc9a-455d-8f4c-8ade321e1cfb admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:32.282757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b57a9-bc9a-455d-8f4c-8ade321e1cfb admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:32.360744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3da08-9a05-484a-9e1a-c8acb08fddd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:32.361712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3da08-9a05-484a-9e1a-c8acb08fddd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:32.361998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3da08-9a05-484a-9e1a-c8acb08fddd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:32.362276 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3da08-9a05-484a-9e1a-c8acb08fddd7 admin admin] PATH_INFO: `/v3/projects/f133d4db3602405dbd1717f77ac9d09c/users/b5a1e90bb5b940c0a8340e1fda48a12a/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:32.365976 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3da08-9a05-484a-9e1a-c8acb08fddd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f133d4db3602405dbd1717f77ac9d09c, user_id=b5a1e90bb5b940c0a8340e1fda48a12a, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:32.371257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da39dc-245c-4486-8720-b59299be7142 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:32.371503 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3da08-9a05-484a-9e1a-c8acb08fddd7 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:32.378392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871ff22-b0f2-4845-8efe-113abbdd40a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:32.379811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871ff22-b0f2-4845-8efe-113abbdd40a8 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:32.380532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871ff22-b0f2-4845-8efe-113abbdd40a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:32.381132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871ff22-b0f2-4845-8efe-113abbdd40a8 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:32.382006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871ff22-b0f2-4845-8efe-113abbdd40a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:32.384398 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871ff22-b0f2-4845-8efe-113abbdd40a8 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:32.387337 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1306/2603] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:32 2022] PUT /identity/v3/projects/f133d4db3602405dbd1717f77ac9d09c/users/b5a1e90bb5b940c0a8340e1fda48a12a/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:32.519687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1298/2604] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:32 2022] POST /identity/v3/users => generated 309 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:32.526436 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3da08-9a05-484a-9e1a-c8acb08fddd7 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:32.643036 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3da08-9a05-484a-9e1a-c8acb08fddd7 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:32.650551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5cdbc-f300-4762-878c-a1b947d57ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:32.657022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5cdbc-f300-4762-878c-a1b947d57ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:32.657022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5cdbc-f300-4762-878c-a1b947d57ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:32.657022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5cdbc-f300-4762-878c-a1b947d57ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:32.657022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d5cdbc-f300-4762-878c-a1b947d57ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:32.657022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d5cdbc-f300-4762-878c-a1b947d57ff4 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:32.671526 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1307/2605] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:32.675373 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871ff22-b0f2-4845-8efe-113abbdd40a8 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:32.690845 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871ff22-b0f2-4845-8efe-113abbdd40a8 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:32.701584 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9765dc-cfec-4c44-8eeb-246145917f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:32.702782 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9765dc-cfec-4c44-8eeb-246145917f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:32.703313 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9765dc-cfec-4c44-8eeb-246145917f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:32.703867 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9765dc-cfec-4c44-8eeb-246145917f2e admin admin] PATH_INFO: `/v3/projects/2ed067f3d693470f8a2764260b47fbfd/users/4db1f614cd334020851c90acf4e0cb4c/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:32.726152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9765dc-cfec-4c44-8eeb-246145917f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed067f3d693470f8a2764260b47fbfd, user_id=4db1f614cd334020851c90acf4e0cb4c, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:32.729354 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9765dc-cfec-4c44-8eeb-246145917f2e admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:32.858099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1299/2606] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:32 2022] PUT /identity/v3/projects/2ed067f3d693470f8a2764260b47fbfd/users/4db1f614cd334020851c90acf4e0cb4c/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:32.862204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5cdbc-f300-4762-878c-a1b947d57ff4 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:32.867623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:32.875160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5cdbc-f300-4762-878c-a1b947d57ff4 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:32.988195 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d865a-3d52-4926-96b9-4725e35893eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:32.989750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d865a-3d52-4926-96b9-4725e35893eb admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:32.990164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d865a-3d52-4926-96b9-4725e35893eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:32.990613 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d865a-3d52-4926-96b9-4725e35893eb admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:32.995126 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d865a-3d52-4926-96b9-4725e35893eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:32.997477 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d865a-3d52-4926-96b9-4725e35893eb admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:33.013079 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1308/2607] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:33.018713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9765dc-cfec-4c44-8eeb-246145917f2e admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:33.052179 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9765dc-cfec-4c44-8eeb-246145917f2e admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:33.063628 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e230d-dcbe-429b-8fcc-73559d0f1d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:33.067081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e230d-dcbe-429b-8fcc-73559d0f1d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:33.067081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e230d-dcbe-429b-8fcc-73559d0f1d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:33.067081 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e230d-dcbe-429b-8fcc-73559d0f1d41 admin admin] PATH_INFO: `/v3/projects/2ed067f3d693470f8a2764260b47fbfd/users/4db1f614cd334020851c90acf4e0cb4c/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:33.072928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e230d-dcbe-429b-8fcc-73559d0f1d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed067f3d693470f8a2764260b47fbfd, user_id=4db1f614cd334020851c90acf4e0cb4c, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:33.077396 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e230d-dcbe-429b-8fcc-73559d0f1d41 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:33.114148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1300/2608] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:33 2022] PUT /identity/v3/projects/2ed067f3d693470f8a2764260b47fbfd/users/4db1f614cd334020851c90acf4e0cb4c/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:33.128872 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e6b36-23da-4031-b3cd-36fb616ecf9d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:33.129263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e6b36-23da-4031-b3cd-36fb616ecf9d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:33.129650 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e6b36-23da-4031-b3cd-36fb616ecf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:33.221056 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-890e6b36-23da-4031-b3cd-36fb616ecf9d None None] MFA Rules not processed for user `4db1f614cd334020851c90acf4e0cb4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:33.221977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-890e6b36-23da-4031-b3cd-36fb616ecf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:33.368779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1309/2609] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:33 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:33.379124 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e230d-dcbe-429b-8fcc-73559d0f1d41 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:33.381139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-295e230d-dcbe-429b-8fcc-73559d0f1d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:33.460330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:33.535054 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e230d-dcbe-429b-8fcc-73559d0f1d41 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:33.549872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc6efc-400d-4cd6-b397-5a1b96c3b3ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:33.549872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc6efc-400d-4cd6-b397-5a1b96c3b3ec service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:33.549872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc6efc-400d-4cd6-b397-5a1b96c3b3ec service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:33.549872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc6efc-400d-4cd6-b397-5a1b96c3b3ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:33.552967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc6efc-400d-4cd6-b397-5a1b96c3b3ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:33.554520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc6efc-400d-4cd6-b397-5a1b96c3b3ec service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:33.566639 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1301/2610] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:33 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:33.748113 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e6b36-23da-4031-b3cd-36fb616ecf9d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:33.840022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e6b36-23da-4031-b3cd-36fb616ecf9d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:33.844979 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69725ce0-3855-4df7-a6e5-782e69293b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:33.845955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69725ce0-3855-4df7-a6e5-782e69293b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:33.846633 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69725ce0-3855-4df7-a6e5-782e69293b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:33.846912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69725ce0-3855-4df7-a6e5-782e69293b92 admin admin] PATH_INFO: `/v3/users/15eaadbcbda540f4a051b9d1fe7c3804` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:33.849616 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69725ce0-3855-4df7-a6e5-782e69293b92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15eaadbcbda540f4a051b9d1fe7c3804)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:33.851581 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69725ce0-3855-4df7-a6e5-782e69293b92 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:34.011310 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:34.168100 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-69725ce0-3855-4df7-a6e5-782e69293b92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15eaadbcbda540f4a051b9d1fe7c3804'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:34.208130 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ae346-95e5-4a7d-a9f3-2e8e94fe43fd None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:34.209316 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ae346-95e5-4a7d-a9f3-2e8e94fe43fd None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:34.209849 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ae346-95e5-4a7d-a9f3-2e8e94fe43fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:34.224422 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-69725ce0-3855-4df7-a6e5-782e69293b92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15eaadbcbda540f4a051b9d1fe7c3804'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:34.291264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-645ae346-95e5-4a7d-a9f3-2e8e94fe43fd None None] MFA Rules not processed for user `b5a1e90bb5b940c0a8340e1fda48a12a`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:34.292643 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-645ae346-95e5-4a7d-a9f3-2e8e94fe43fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:34.330981 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-69725ce0-3855-4df7-a6e5-782e69293b92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15eaadbcbda540f4a051b9d1fe7c3804'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:34.356073 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-69725ce0-3855-4df7-a6e5-782e69293b92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15eaadbcbda540f4a051b9d1fe7c3804'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:34.357687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-69725ce0-3855-4df7-a6e5-782e69293b92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15eaadbcbda540f4a051b9d1fe7c3804'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:34.374502 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1310/2611] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:33 2022] DELETE /identity/v3/users/15eaadbcbda540f4a051b9d1fe7c3804 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:34.382169 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707305e1-dd40-4a08-b2ad-e4a9e3e84ef8 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:34.382520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707305e1-dd40-4a08-b2ad-e4a9e3e84ef8 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:34.382934 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707305e1-dd40-4a08-b2ad-e4a9e3e84ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:34.413841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-707305e1-dd40-4a08-b2ad-e4a9e3e84ef8 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:34.416125 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-707305e1-dd40-4a08-b2ad-e4a9e3e84ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:34.437433 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1302/2612] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:34 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:13:34.454713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ae346-95e5-4a7d-a9f3-2e8e94fe43fd None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:34.459378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-645ae346-95e5-4a7d-a9f3-2e8e94fe43fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:34.486166 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ae346-95e5-4a7d-a9f3-2e8e94fe43fd None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:34.553012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1311/2613] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:34.563651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707305e1-dd40-4a08-b2ad-e4a9e3e84ef8 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:34.566233 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-707305e1-dd40-4a08-b2ad-e4a9e3e84ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:34.582113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e76d6-23d9-4782-ba2e-f41aec56a8eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:34.583363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e76d6-23d9-4782-ba2e-f41aec56a8eb service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:34.583853 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e76d6-23d9-4782-ba2e-f41aec56a8eb service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:34.584386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e76d6-23d9-4782-ba2e-f41aec56a8eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:34.587615 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-650e76d6-23d9-4782-ba2e-f41aec56a8eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:34.670967 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e76d6-23d9-4782-ba2e-f41aec56a8eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:34.673768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e76d6-23d9-4782-ba2e-f41aec56a8eb service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:34.681792 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707305e1-dd40-4a08-b2ad-e4a9e3e84ef8 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:34.686626 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4095d-059b-4e0a-961a-3505611400b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:34.687796 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4095d-059b-4e0a-961a-3505611400b9 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:34.688174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4095d-059b-4e0a-961a-3505611400b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:34.688546 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4095d-059b-4e0a-961a-3505611400b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:34.699653 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1303/2614] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:34.700935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4095d-059b-4e0a-961a-3505611400b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:34.702513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4095d-059b-4e0a-961a-3505611400b9 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:34.717329 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1312/2615] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:35.279408 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e76d6-23d9-4782-ba2e-f41aec56a8eb service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:35.279408 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-650e76d6-23d9-4782-ba2e-f41aec56a8eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:35.300049 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e76d6-23d9-4782-ba2e-f41aec56a8eb service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:35.305175 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:35.307695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:35.307695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:35.307695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] PATH_INFO: `/v3/projects/5ada2790f9fc4118b4cefbcc7b8d1884` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:35.311535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ada2790f9fc4118b4cefbcc7b8d1884)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:35.316503 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:35.322508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:35.513792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ada2790f9fc4118b4cefbcc7b8d1884', 'request_id': 'req-ad964d3a-a35f-4829-ba5e-07a901e21ed5'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:35.523138 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ada2790f9fc4118b4cefbcc7b8d1884', 'request_id': 'req-ad964d3a-a35f-4829-ba5e-07a901e21ed5'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:35.531707 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ada2790f9fc4118b4cefbcc7b8d1884', 'request_id': 'req-ad964d3a-a35f-4829-ba5e-07a901e21ed5'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:35.952924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] The token cache is being invalidate because project 5ada2790f9fc4118b4cefbcc7b8d1884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:35.965944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:35.965944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:35.965944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1304/2616] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:35 2022] DELETE /identity/v3/projects/5ada2790f9fc4118b4cefbcc7b8d1884 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:35.968956 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4095d-059b-4e0a-961a-3505611400b9 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:35.979483 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4095d-059b-4e0a-961a-3505611400b9 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:36.033267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:36.045968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673be685-5791-4f68-979a-e03440cda745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:36.046612 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673be685-5791-4f68-979a-e03440cda745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:36.046885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673be685-5791-4f68-979a-e03440cda745 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:36.047156 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673be685-5791-4f68-979a-e03440cda745 admin admin] PATH_INFO: `/v3/users/4db1f614cd334020851c90acf4e0cb4c` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:36.048859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673be685-5791-4f68-979a-e03440cda745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db1f614cd334020851c90acf4e0cb4c)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:36.051103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673be685-5791-4f68-979a-e03440cda745 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:36.428462 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-673be685-5791-4f68-979a-e03440cda745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db1f614cd334020851c90acf4e0cb4c'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:36.439080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-673be685-5791-4f68-979a-e03440cda745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db1f614cd334020851c90acf4e0cb4c'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:36.558969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-673be685-5791-4f68-979a-e03440cda745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db1f614cd334020851c90acf4e0cb4c'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:36.577930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-673be685-5791-4f68-979a-e03440cda745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db1f614cd334020851c90acf4e0cb4c'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:36.578700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-673be685-5791-4f68-979a-e03440cda745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db1f614cd334020851c90acf4e0cb4c'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:36.589260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1313/2617] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:35 2022] DELETE /identity/v3/users/4db1f614cd334020851c90acf4e0cb4c => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:37.168878 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:37.170976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:37.188274 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad964d3a-a35f-4829-ba5e-07a901e21ed5 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:37.309412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c220645-f2dc-4d3b-a2b8-54f2abe73184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:37.310171 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c220645-f2dc-4d3b-a2b8-54f2abe73184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:37.310601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c220645-f2dc-4d3b-a2b8-54f2abe73184 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:37.310898 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c220645-f2dc-4d3b-a2b8-54f2abe73184 admin admin] PATH_INFO: `/v3/projects/2ed067f3d693470f8a2764260b47fbfd` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:37.312836 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c220645-f2dc-4d3b-a2b8-54f2abe73184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ed067f3d693470f8a2764260b47fbfd)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:37.315207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c220645-f2dc-4d3b-a2b8-54f2abe73184 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:37.491162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7c220645-f2dc-4d3b-a2b8-54f2abe73184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ed067f3d693470f8a2764260b47fbfd', 'request_id': 'req-7c220645-f2dc-4d3b-a2b8-54f2abe73184'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:37.499191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7c220645-f2dc-4d3b-a2b8-54f2abe73184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ed067f3d693470f8a2764260b47fbfd', 'request_id': 'req-7c220645-f2dc-4d3b-a2b8-54f2abe73184'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:37.507516 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7c220645-f2dc-4d3b-a2b8-54f2abe73184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ed067f3d693470f8a2764260b47fbfd', 'request_id': 'req-7c220645-f2dc-4d3b-a2b8-54f2abe73184'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:37.603790 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7c220645-f2dc-4d3b-a2b8-54f2abe73184 admin admin] The token cache is being invalidate because project 2ed067f3d693470f8a2764260b47fbfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:37.604212 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7c220645-f2dc-4d3b-a2b8-54f2abe73184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:37.608174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7c220645-f2dc-4d3b-a2b8-54f2abe73184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:37.613271 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1305/2618] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:37 2022] DELETE /identity/v3/projects/2ed067f3d693470f8a2764260b47fbfd => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:37.619179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e566530-b8c8-4bc0-83b5-90da2f01b2b9 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:37.620077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e566530-b8c8-4bc0-83b5-90da2f01b2b9 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:37.620077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e566530-b8c8-4bc0-83b5-90da2f01b2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:37.665183 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-3e566530-b8c8-4bc0-83b5-90da2f01b2b9 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:37.667080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-3e566530-b8c8-4bc0-83b5-90da2f01b2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:37.750411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1314/2619] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:37.753715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccad921-25d9-4c90-8fa9-a95c3b3f280a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:37.754006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccad921-25d9-4c90-8fa9-a95c3b3f280a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:37.754267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccad921-25d9-4c90-8fa9-a95c3b3f280a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:37.783370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-9ccad921-25d9-4c90-8fa9-a95c3b3f280a None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:37.785107 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9ccad921-25d9-4c90-8fa9-a95c3b3f280a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:37.798524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1306/2620] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:37.799437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e566530-b8c8-4bc0-83b5-90da2f01b2b9 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:37.818950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccad921-25d9-4c90-8fa9-a95c3b3f280a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:37.820881 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9ccad921-25d9-4c90-8fa9-a95c3b3f280a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:37.827932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e566530-b8c8-4bc0-83b5-90da2f01b2b9 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:37.842055 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaa89e-c433-4e5f-b331-27466991523e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:37.843048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaa89e-c433-4e5f-b331-27466991523e admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:37.844239 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaa89e-c433-4e5f-b331-27466991523e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:37.844533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaa89e-c433-4e5f-b331-27466991523e admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:37.845256 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaa89e-c433-4e5f-b331-27466991523e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:13:37.845566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaa89e-c433-4e5f-b331-27466991523e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:37.847975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaa89e-c433-4e5f-b331-27466991523e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:37.851994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:37.877468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1315/2621] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:13:37.879323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaa89e-c433-4e5f-b331-27466991523e admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:37.896623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaa89e-c433-4e5f-b331-27466991523e admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:37.904344 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b31e3e-b1ef-4b77-bc16-1d7587eb5d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:37.907496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b31e3e-b1ef-4b77-bc16-1d7587eb5d59 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:37.907496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b31e3e-b1ef-4b77-bc16-1d7587eb5d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:37.907496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b31e3e-b1ef-4b77-bc16-1d7587eb5d59 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:37.907496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b31e3e-b1ef-4b77-bc16-1d7587eb5d59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:37.908913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b31e3e-b1ef-4b77-bc16-1d7587eb5d59 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:37.917619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccad921-25d9-4c90-8fa9-a95c3b3f280a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:37.922531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145edacd-1f69-4320-9532-4e6429f56544 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:37.923658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145edacd-1f69-4320-9532-4e6429f56544 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:37.924012 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145edacd-1f69-4320-9532-4e6429f56544 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:37.924366 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145edacd-1f69-4320-9532-4e6429f56544 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:37.926703 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-145edacd-1f69-4320-9532-4e6429f56544 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:38.010024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145edacd-1f69-4320-9532-4e6429f56544 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:38.011715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145edacd-1f69-4320-9532-4e6429f56544 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:38.030042 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1307/2622] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:37 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:13:38.041939 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1316/2623] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:37 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:38.047833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145edacd-1f69-4320-9532-4e6429f56544 service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:38.068588 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145edacd-1f69-4320-9532-4e6429f56544 service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:38.155629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28269b5-9632-46c6-85eb-c5cfbcce8942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:38.156216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28269b5-9632-46c6-85eb-c5cfbcce8942 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:38.156501 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28269b5-9632-46c6-85eb-c5cfbcce8942 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:38.156778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28269b5-9632-46c6-85eb-c5cfbcce8942 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:38.157459 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28269b5-9632-46c6-85eb-c5cfbcce8942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:38.159871 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28269b5-9632-46c6-85eb-c5cfbcce8942 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:38.258207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1308/2624] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:38 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:38.263530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b31e3e-b1ef-4b77-bc16-1d7587eb5d59 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:38.279272 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b31e3e-b1ef-4b77-bc16-1d7587eb5d59 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:38.287064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6700f4-aae5-4a87-bdfd-5c49d6530a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:38.287943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6700f4-aae5-4a87-bdfd-5c49d6530a55 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:38.288303 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6700f4-aae5-4a87-bdfd-5c49d6530a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:38.288862 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6700f4-aae5-4a87-bdfd-5c49d6530a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:38.290065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6700f4-aae5-4a87-bdfd-5c49d6530a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:38.292465 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6700f4-aae5-4a87-bdfd-5c49d6530a55 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:38.311033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1317/2625] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:38.312949 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28269b5-9632-46c6-85eb-c5cfbcce8942 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:38.328184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28269b5-9632-46c6-85eb-c5cfbcce8942 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:38.334424 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caeed88-e0c4-40f2-8674-4cc04bc2f321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:38.335082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeed88-e0c4-40f2-8674-4cc04bc2f321 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:38.335364 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeed88-e0c4-40f2-8674-4cc04bc2f321 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:38.335677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeed88-e0c4-40f2-8674-4cc04bc2f321 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:38.336394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caeed88-e0c4-40f2-8674-4cc04bc2f321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:38.339142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caeed88-e0c4-40f2-8674-4cc04bc2f321 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:38.356440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1309/2626] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:38.364365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6700f4-aae5-4a87-bdfd-5c49d6530a55 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:38.394849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6700f4-aae5-4a87-bdfd-5c49d6530a55 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:38.407211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a588a9-74e5-484b-870b-20c48a569616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:38.408689 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a588a9-74e5-484b-870b-20c48a569616 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:38.409094 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a588a9-74e5-484b-870b-20c48a569616 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:38.409502 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a588a9-74e5-484b-870b-20c48a569616 admin admin] PATH_INFO: `/v3/projects/65f5b9856565473c956c69501b8b2f10/users/b38f1136933a487fa7c0d93ae07d80fd/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:38.446176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a588a9-74e5-484b-870b-20c48a569616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f5b9856565473c956c69501b8b2f10, user_id=b38f1136933a487fa7c0d93ae07d80fd, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:38.454378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a588a9-74e5-484b-870b-20c48a569616 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:38.463018 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:38.603099 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1318/2627] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:38 2022] PUT /identity/v3/projects/65f5b9856565473c956c69501b8b2f10/users/b38f1136933a487fa7c0d93ae07d80fd/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:38.637527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc39f1-5c07-48b7-946d-c8d437c2c694 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:38.637795 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc39f1-5c07-48b7-946d-c8d437c2c694 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:38.638065 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc39f1-5c07-48b7-946d-c8d437c2c694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:38.698369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ccdc39f1-5c07-48b7-946d-c8d437c2c694 None None] MFA Rules not processed for user `b38f1136933a487fa7c0d93ae07d80fd`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:38.699728 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ccdc39f1-5c07-48b7-946d-c8d437c2c694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:38.741846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:38.798556 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1310/2628] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:13:38.800032 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a588a9-74e5-484b-870b-20c48a569616 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:38.918204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a588a9-74e5-484b-870b-20c48a569616 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:38.923316 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072f819-7684-402a-a783-f39cb0404309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:38.923993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072f819-7684-402a-a783-f39cb0404309 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:38.924494 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072f819-7684-402a-a783-f39cb0404309 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:38.924619 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072f819-7684-402a-a783-f39cb0404309 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:38.925415 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072f819-7684-402a-a783-f39cb0404309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:38.927560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072f819-7684-402a-a783-f39cb0404309 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:39.033479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1319/2629] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:38 2022] POST /identity/v3/projects => generated 365 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:39.035118 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc39f1-5c07-48b7-946d-c8d437c2c694 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:39.050008 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc39f1-5c07-48b7-946d-c8d437c2c694 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:39.128570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdd30c-18ca-45bb-8b7d-a8e28200835c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:39.129272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdd30c-18ca-45bb-8b7d-a8e28200835c admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:39.129532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdd30c-18ca-45bb-8b7d-a8e28200835c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:39.129811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdd30c-18ca-45bb-8b7d-a8e28200835c admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:39.130507 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcdd30c-18ca-45bb-8b7d-a8e28200835c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:39.132581 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcdd30c-18ca-45bb-8b7d-a8e28200835c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:39.295738 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1311/2630] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:39 2022] POST /identity/v3/users => generated 289 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:39.300298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072f819-7684-402a-a783-f39cb0404309 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:39.313181 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072f819-7684-402a-a783-f39cb0404309 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:39.319435 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c530716-cfb3-4694-b23b-07d61fe58c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:39.320104 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c530716-cfb3-4694-b23b-07d61fe58c44 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:39.320380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c530716-cfb3-4694-b23b-07d61fe58c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:39.320645 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c530716-cfb3-4694-b23b-07d61fe58c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:39.322219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c530716-cfb3-4694-b23b-07d61fe58c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:39.324258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c530716-cfb3-4694-b23b-07d61fe58c44 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:39.337974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1320/2631] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:13:39.342201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdd30c-18ca-45bb-8b7d-a8e28200835c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:39.359234 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdd30c-18ca-45bb-8b7d-a8e28200835c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:39.367350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16acd9-e422-4113-a0dc-ab3368997735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:39.369480 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16acd9-e422-4113-a0dc-ab3368997735 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:39.370308 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16acd9-e422-4113-a0dc-ab3368997735 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:39.370921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16acd9-e422-4113-a0dc-ab3368997735 admin admin] PATH_INFO: `/v3/projects/660081e9e1914771a47b3caf9fcc2923/users/934f1674ded243129773993445e6eab5/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:39.393076 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16acd9-e422-4113-a0dc-ab3368997735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=660081e9e1914771a47b3caf9fcc2923, user_id=934f1674ded243129773993445e6eab5, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:39.395813 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16acd9-e422-4113-a0dc-ab3368997735 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:39.958761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1312/2632] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:39 2022] PUT /identity/v3/projects/660081e9e1914771a47b3caf9fcc2923/users/934f1674ded243129773993445e6eab5/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:39.963273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c530716-cfb3-4694-b23b-07d61fe58c44 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:39.973623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c530716-cfb3-4694-b23b-07d61fe58c44 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:40.033377 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c21b1c-b036-473d-a16c-14f6a3c1d02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:40.034000 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c21b1c-b036-473d-a16c-14f6a3c1d02d admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:40.034282 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c21b1c-b036-473d-a16c-14f6a3c1d02d admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:40.034557 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c21b1c-b036-473d-a16c-14f6a3c1d02d admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:40.035282 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c21b1c-b036-473d-a16c-14f6a3c1d02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:40.036935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c21b1c-b036-473d-a16c-14f6a3c1d02d admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:40.048338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1321/2633] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:40.052647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16acd9-e422-4113-a0dc-ab3368997735 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:40.067155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16acd9-e422-4113-a0dc-ab3368997735 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:40.070302 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710fe1a-7330-4592-ac57-05e97b8f8302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:40.071393 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710fe1a-7330-4592-ac57-05e97b8f8302 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:40.071786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710fe1a-7330-4592-ac57-05e97b8f8302 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:40.072066 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710fe1a-7330-4592-ac57-05e97b8f8302 admin admin] PATH_INFO: `/v3/projects/660081e9e1914771a47b3caf9fcc2923/users/934f1674ded243129773993445e6eab5/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:40.074117 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710fe1a-7330-4592-ac57-05e97b8f8302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=660081e9e1914771a47b3caf9fcc2923, user_id=934f1674ded243129773993445e6eab5, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:40.077094 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710fe1a-7330-4592-ac57-05e97b8f8302 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:40.089914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1313/2634] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:40 2022] PUT /identity/v3/projects/660081e9e1914771a47b3caf9fcc2923/users/934f1674ded243129773993445e6eab5/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:40.103132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465bd81-0826-4031-b116-c6f9d5fef367 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:40.103412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465bd81-0826-4031-b116-c6f9d5fef367 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:40.103720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465bd81-0826-4031-b116-c6f9d5fef367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:40.142450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-a465bd81-0826-4031-b116-c6f9d5fef367 None None] MFA Rules not processed for user `934f1674ded243129773993445e6eab5`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:40.143877 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a465bd81-0826-4031-b116-c6f9d5fef367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:40.209707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:40.231216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1322/2635] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:40 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:40.250504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710fe1a-7330-4592-ac57-05e97b8f8302 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:40.323416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710fe1a-7330-4592-ac57-05e97b8f8302 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:40.328328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729d7f8-54d5-47a0-997f-9be4e9563b55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:40.329318 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729d7f8-54d5-47a0-997f-9be4e9563b55 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:40.329717 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729d7f8-54d5-47a0-997f-9be4e9563b55 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:40.330304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729d7f8-54d5-47a0-997f-9be4e9563b55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:40.341858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729d7f8-54d5-47a0-997f-9be4e9563b55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:40.343510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729d7f8-54d5-47a0-997f-9be4e9563b55 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:40.374573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1314/2636] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:40 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:40.657949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465bd81-0826-4031-b116-c6f9d5fef367 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:40.678251 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465bd81-0826-4031-b116-c6f9d5fef367 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:40.684911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f04811-1ff9-4021-b444-275fba67be9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:40.685904 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f04811-1ff9-4021-b444-275fba67be9b service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:40.686416 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f04811-1ff9-4021-b444-275fba67be9b service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:40.686780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f04811-1ff9-4021-b444-275fba67be9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:40.761915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f04811-1ff9-4021-b444-275fba67be9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:40.763527 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f04811-1ff9-4021-b444-275fba67be9b service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:40.777061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1323/2637] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:45.057009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729d7f8-54d5-47a0-997f-9be4e9563b55 service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:45.091949 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:45.124858 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729d7f8-54d5-47a0-997f-9be4e9563b55 service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:45.129498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668e1f6-31bc-4c4e-b6ed-9ba333d6268b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:45.130127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668e1f6-31bc-4c4e-b6ed-9ba333d6268b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:45.130410 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668e1f6-31bc-4c4e-b6ed-9ba333d6268b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:45.130688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668e1f6-31bc-4c4e-b6ed-9ba333d6268b admin admin] PATH_INFO: `/v3/users/b38f1136933a487fa7c0d93ae07d80fd` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:45.131921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668e1f6-31bc-4c4e-b6ed-9ba333d6268b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b38f1136933a487fa7c0d93ae07d80fd)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:45.133926 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668e1f6-31bc-4c4e-b6ed-9ba333d6268b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:45.357530 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b668e1f6-31bc-4c4e-b6ed-9ba333d6268b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b38f1136933a487fa7c0d93ae07d80fd'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:45.360277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b668e1f6-31bc-4c4e-b6ed-9ba333d6268b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b38f1136933a487fa7c0d93ae07d80fd'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:45.360277 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b668e1f6-31bc-4c4e-b6ed-9ba333d6268b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b38f1136933a487fa7c0d93ae07d80fd'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:45.368241 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b668e1f6-31bc-4c4e-b6ed-9ba333d6268b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b38f1136933a487fa7c0d93ae07d80fd'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:45.381562 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b668e1f6-31bc-4c4e-b6ed-9ba333d6268b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b38f1136933a487fa7c0d93ae07d80fd'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:45.506267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1315/2638] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:45 2022] DELETE /identity/v3/users/b38f1136933a487fa7c0d93ae07d80fd => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:45.511584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ff247-a4d9-41a4-b551-7bb2eca3569d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:45.512097 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ff247-a4d9-41a4-b551-7bb2eca3569d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:45.512218 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ff247-a4d9-41a4-b551-7bb2eca3569d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:45.532135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-197ff247-a4d9-41a4-b551-7bb2eca3569d None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:45.533364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-197ff247-a4d9-41a4-b551-7bb2eca3569d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:45.598902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1324/2639] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:45.627545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668e1f6-31bc-4c4e-b6ed-9ba333d6268b admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:45.629014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b668e1f6-31bc-4c4e-b6ed-9ba333d6268b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:45.694599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668e1f6-31bc-4c4e-b6ed-9ba333d6268b admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:45.698921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461272a-c2f6-4985-9d17-d409ebc3bb6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:45.699617 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461272a-c2f6-4985-9d17-d409ebc3bb6b service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:45.700236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461272a-c2f6-4985-9d17-d409ebc3bb6b service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:45.700236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461272a-c2f6-4985-9d17-d409ebc3bb6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:45.709917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461272a-c2f6-4985-9d17-d409ebc3bb6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:45.711682 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461272a-c2f6-4985-9d17-d409ebc3bb6b service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:45.722230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1316/2640] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:46.373386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ff247-a4d9-41a4-b551-7bb2eca3569d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:46.374726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-197ff247-a4d9-41a4-b551-7bb2eca3569d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:46.390678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ff247-a4d9-41a4-b551-7bb2eca3569d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:46.393697 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:46.394350 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:46.394632 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:46.394908 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] PATH_INFO: `/v3/projects/65f5b9856565473c956c69501b8b2f10` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:46.397193 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] RBAC: Authorizing `identity:delete_project(project_id=65f5b9856565473c956c69501b8b2f10)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:46.399493 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:46.562291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65f5b9856565473c956c69501b8b2f10', 'request_id': 'req-d71a68f5-94c8-490c-a9a0-be26e917123e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:46.568713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65f5b9856565473c956c69501b8b2f10', 'request_id': 'req-d71a68f5-94c8-490c-a9a0-be26e917123e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:46.660520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65f5b9856565473c956c69501b8b2f10', 'request_id': 'req-d71a68f5-94c8-490c-a9a0-be26e917123e'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:46.674051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] The token cache is being invalidate because project 65f5b9856565473c956c69501b8b2f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:46.674354 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:46.675024 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:46.676598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1325/2641] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:46 2022] DELETE /identity/v3/projects/65f5b9856565473c956c69501b8b2f10 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:46.682079 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461272a-c2f6-4985-9d17-d409ebc3bb6b service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:46.693247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461272a-c2f6-4985-9d17-d409ebc3bb6b service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:46.869078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:46.879998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae80985-6bc9-4f17-b7d2-a0e9b8b2f591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:46.880635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae80985-6bc9-4f17-b7d2-a0e9b8b2f591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:46.880913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae80985-6bc9-4f17-b7d2-a0e9b8b2f591 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:46.881207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae80985-6bc9-4f17-b7d2-a0e9b8b2f591 admin admin] PATH_INFO: `/v3/users/934f1674ded243129773993445e6eab5` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:46.882489 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae80985-6bc9-4f17-b7d2-a0e9b8b2f591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=934f1674ded243129773993445e6eab5)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:46.884484 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae80985-6bc9-4f17-b7d2-a0e9b8b2f591 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:47.140021 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7ae80985-6bc9-4f17-b7d2-a0e9b8b2f591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934f1674ded243129773993445e6eab5'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:47.140782 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7ae80985-6bc9-4f17-b7d2-a0e9b8b2f591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934f1674ded243129773993445e6eab5'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:47.141460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7ae80985-6bc9-4f17-b7d2-a0e9b8b2f591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934f1674ded243129773993445e6eab5'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:47.151405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7ae80985-6bc9-4f17-b7d2-a0e9b8b2f591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934f1674ded243129773993445e6eab5'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:47.165216 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7ae80985-6bc9-4f17-b7d2-a0e9b8b2f591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934f1674ded243129773993445e6eab5'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:47.272632 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1317/2642] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:46 2022] DELETE /identity/v3/users/934f1674ded243129773993445e6eab5 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:47.933468 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:47.935495 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:47.953027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a68f5-94c8-490c-a9a0-be26e917123e admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:47.965305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:48.020196 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:48.020842 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:48.021118 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:48.021385 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41 admin admin] PATH_INFO: `/v3/projects/660081e9e1914771a47b3caf9fcc2923` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:48.022943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=660081e9e1914771a47b3caf9fcc2923)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:48.025076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:48.177076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '660081e9e1914771a47b3caf9fcc2923', 'request_id': 'req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:48.183785 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '660081e9e1914771a47b3caf9fcc2923', 'request_id': 'req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:48.290888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '660081e9e1914771a47b3caf9fcc2923', 'request_id': 'req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:48.308071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41 admin admin] The token cache is being invalidate because project 660081e9e1914771a47b3caf9fcc2923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:48.308454 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:48.309442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d6e3fdf4-52ab-4d27-9399-dd330b73eb41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:48.311735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1326/2643] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:47 2022] DELETE /identity/v3/projects/660081e9e1914771a47b3caf9fcc2923 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:48.317189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b597305-ec0f-4814-ab7a-3693827d9fa3 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:48.317699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b597305-ec0f-4814-ab7a-3693827d9fa3 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:48.322970 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b597305-ec0f-4814-ab7a-3693827d9fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:48.338742 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-2b597305-ec0f-4814-ab7a-3693827d9fa3 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:48.340031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-2b597305-ec0f-4814-ab7a-3693827d9fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:48.432391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1318/2644] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:48.440591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d728c7c-abdb-4539-9f9c-868f9934ea5c None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:48.440972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d728c7c-abdb-4539-9f9c-868f9934ea5c None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:48.441335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d728c7c-abdb-4539-9f9c-868f9934ea5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:48.463895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7d728c7c-abdb-4539-9f9c-868f9934ea5c None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:48.465140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7d728c7c-abdb-4539-9f9c-868f9934ea5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:48.477027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1327/2645] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:48.478404 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b597305-ec0f-4814-ab7a-3693827d9fa3 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:48.494613 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b597305-ec0f-4814-ab7a-3693827d9fa3 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:48.500172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a7d51-7179-479e-9963-2212ee01ad64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:48.504178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a7d51-7179-479e-9963-2212ee01ad64 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:48.504471 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a7d51-7179-479e-9963-2212ee01ad64 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:48.504797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a7d51-7179-479e-9963-2212ee01ad64 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:48.505476 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a7d51-7179-479e-9963-2212ee01ad64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:13:48.505783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a7d51-7179-479e-9963-2212ee01ad64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:48.507386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a7d51-7179-479e-9963-2212ee01ad64 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:48.529086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1319/2646] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:13:48.533684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d728c7c-abdb-4539-9f9c-868f9934ea5c None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:48.551300 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d728c7c-abdb-4539-9f9c-868f9934ea5c None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:48.561210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176da908-48b2-4af5-972b-62ffb033b0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:48.562248 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176da908-48b2-4af5-972b-62ffb033b0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:48.563628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176da908-48b2-4af5-972b-62ffb033b0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:48.564235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176da908-48b2-4af5-972b-62ffb033b0b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:48.565684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176da908-48b2-4af5-972b-62ffb033b0b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:48.567773 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176da908-48b2-4af5-972b-62ffb033b0b4 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:48.695040 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1328/2647] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:48 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:48.697415 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a7d51-7179-479e-9963-2212ee01ad64 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:48.707992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a7d51-7179-479e-9963-2212ee01ad64 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:48.762270 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:48.768260 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42dec3-ba62-4c58-992b-6a157b37a69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:48.768903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42dec3-ba62-4c58-992b-6a157b37a69b admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:48.769189 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42dec3-ba62-4c58-992b-6a157b37a69b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:48.769466 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42dec3-ba62-4c58-992b-6a157b37a69b admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:48.770134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42dec3-ba62-4c58-992b-6a157b37a69b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:48.772161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42dec3-ba62-4c58-992b-6a157b37a69b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:48.901226 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1320/2648] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:48 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:48.905553 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176da908-48b2-4af5-972b-62ffb033b0b4 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:48.917263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176da908-48b2-4af5-972b-62ffb033b0b4 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:48.922517 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8549a1-e8c2-47df-925d-f3ba4fe2fa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:48.923136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8549a1-e8c2-47df-925d-f3ba4fe2fa10 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:48.923415 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8549a1-e8c2-47df-925d-f3ba4fe2fa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:48.923719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8549a1-e8c2-47df-925d-f3ba4fe2fa10 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:48.924412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8549a1-e8c2-47df-925d-f3ba4fe2fa10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:48.926229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8549a1-e8c2-47df-925d-f3ba4fe2fa10 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:48.936673 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1329/2649] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:48.940869 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42dec3-ba62-4c58-992b-6a157b37a69b admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:48.953162 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42dec3-ba62-4c58-992b-6a157b37a69b admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:48.958933 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57e648-bda6-483d-b8b5-406c50366df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:48.959699 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57e648-bda6-483d-b8b5-406c50366df0 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:48.960856 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57e648-bda6-483d-b8b5-406c50366df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:48.960856 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57e648-bda6-483d-b8b5-406c50366df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:48.962113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57e648-bda6-483d-b8b5-406c50366df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:48.963792 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57e648-bda6-483d-b8b5-406c50366df0 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:48.974623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1321/2650] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:48.979168 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8549a1-e8c2-47df-925d-f3ba4fe2fa10 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:48.990894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8549a1-e8c2-47df-925d-f3ba4fe2fa10 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:48.999306 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a2cb6-0121-4556-a2d2-ac3178c378e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:49.000023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a2cb6-0121-4556-a2d2-ac3178c378e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:49.000304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a2cb6-0121-4556-a2d2-ac3178c378e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:49.000616 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a2cb6-0121-4556-a2d2-ac3178c378e6 admin admin] PATH_INFO: `/v3/projects/039ae51ef6364be380785b7ab1f1bfc1/users/359d68c3c6be4e30817e28cd0be342c8/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:49.017220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a2cb6-0121-4556-a2d2-ac3178c378e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=039ae51ef6364be380785b7ab1f1bfc1, user_id=359d68c3c6be4e30817e28cd0be342c8, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:49.020160 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a2cb6-0121-4556-a2d2-ac3178c378e6 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:49.126665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1330/2651] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:48 2022] PUT /identity/v3/projects/039ae51ef6364be380785b7ab1f1bfc1/users/359d68c3c6be4e30817e28cd0be342c8/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:49.138931 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6d7e9-cc62-4ef7-ae25-3d96923d518d None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:49.139213 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6d7e9-cc62-4ef7-ae25-3d96923d518d None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:49.139525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6d7e9-cc62-4ef7-ae25-3d96923d518d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:49.170132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-89b6d7e9-cc62-4ef7-ae25-3d96923d518d None None] MFA Rules not processed for user `359d68c3c6be4e30817e28cd0be342c8`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:49.171352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-89b6d7e9-cc62-4ef7-ae25-3d96923d518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:49.248914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1322/2652] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:49.250041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a2cb6-0121-4556-a2d2-ac3178c378e6 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:49.298546 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:49.318412 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a2cb6-0121-4556-a2d2-ac3178c378e6 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:49.324155 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd83c7-965a-4e67-90b4-5146221c8166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:49.324780 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd83c7-965a-4e67-90b4-5146221c8166 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:49.325059 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd83c7-965a-4e67-90b4-5146221c8166 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:49.325327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd83c7-965a-4e67-90b4-5146221c8166 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:49.325994 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd83c7-965a-4e67-90b4-5146221c8166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:49.328012 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd83c7-965a-4e67-90b4-5146221c8166 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:49.538206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1331/2653] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:49 2022] POST /identity/v3/projects => generated 383 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:49.543346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6d7e9-cc62-4ef7-ae25-3d96923d518d None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:49.555641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6d7e9-cc62-4ef7-ae25-3d96923d518d None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:49.615750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7507a3d-5d40-4a33-8976-65f04f834bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:49.616370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7507a3d-5d40-4a33-8976-65f04f834bff admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:49.616654 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7507a3d-5d40-4a33-8976-65f04f834bff admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:49.616927 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7507a3d-5d40-4a33-8976-65f04f834bff admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:49.618246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7507a3d-5d40-4a33-8976-65f04f834bff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:49.620254 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7507a3d-5d40-4a33-8976-65f04f834bff admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:49.746419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1323/2654] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:49 2022] POST /identity/v3/users => generated 298 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:49.750457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd83c7-965a-4e67-90b4-5146221c8166 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:49.761687 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd83c7-965a-4e67-90b4-5146221c8166 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:49.766170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb4181-dee9-4d85-ada4-1f8f5eec7072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:49.766783 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb4181-dee9-4d85-ada4-1f8f5eec7072 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:49.767063 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb4181-dee9-4d85-ada4-1f8f5eec7072 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:49.767334 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb4181-dee9-4d85-ada4-1f8f5eec7072 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:49.768105 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb4181-dee9-4d85-ada4-1f8f5eec7072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:49.769678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb4181-dee9-4d85-ada4-1f8f5eec7072 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:49.780684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1332/2655] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:49.785007 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7507a3d-5d40-4a33-8976-65f04f834bff admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:49.796146 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7507a3d-5d40-4a33-8976-65f04f834bff admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:49.800816 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a9dc0-e5d7-4f55-af76-29808392bbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:49.801521 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a9dc0-e5d7-4f55-af76-29808392bbe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:49.801807 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a9dc0-e5d7-4f55-af76-29808392bbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:49.802086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a9dc0-e5d7-4f55-af76-29808392bbe9 admin admin] PATH_INFO: `/v3/projects/2b25677e9b62477aba9adfc897712a77/users/256f7cc723aa48fabd9be155e0c7f057/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:49.817112 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a9dc0-e5d7-4f55-af76-29808392bbe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b25677e9b62477aba9adfc897712a77, user_id=256f7cc723aa48fabd9be155e0c7f057, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:49.819845 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a9dc0-e5d7-4f55-af76-29808392bbe9 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:49.904634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1324/2656] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:49 2022] PUT /identity/v3/projects/2b25677e9b62477aba9adfc897712a77/users/256f7cc723aa48fabd9be155e0c7f057/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:49.908700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb4181-dee9-4d85-ada4-1f8f5eec7072 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:49.928102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb4181-dee9-4d85-ada4-1f8f5eec7072 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:49.990434 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d00d27-f787-4542-a64e-f0848b3b277d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:49.991244 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d00d27-f787-4542-a64e-f0848b3b277d admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:49.991336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d00d27-f787-4542-a64e-f0848b3b277d admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:49.991812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d00d27-f787-4542-a64e-f0848b3b277d admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:49.992456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d00d27-f787-4542-a64e-f0848b3b277d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:49.994142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d00d27-f787-4542-a64e-f0848b3b277d admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:50.009758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1333/2657] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:50.013640 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a9dc0-e5d7-4f55-af76-29808392bbe9 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:50.025701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a9dc0-e5d7-4f55-af76-29808392bbe9 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:50.030436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd29fef-207c-4061-a6bf-4f337b5ac8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:50.031210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd29fef-207c-4061-a6bf-4f337b5ac8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:50.031527 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd29fef-207c-4061-a6bf-4f337b5ac8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:50.031805 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd29fef-207c-4061-a6bf-4f337b5ac8b2 admin admin] PATH_INFO: `/v3/projects/2b25677e9b62477aba9adfc897712a77/users/256f7cc723aa48fabd9be155e0c7f057/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:50.033803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd29fef-207c-4061-a6bf-4f337b5ac8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b25677e9b62477aba9adfc897712a77, user_id=256f7cc723aa48fabd9be155e0c7f057, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:50.036480 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd29fef-207c-4061-a6bf-4f337b5ac8b2 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:50.048126 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1325/2658] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:50 2022] PUT /identity/v3/projects/2b25677e9b62477aba9adfc897712a77/users/256f7cc723aa48fabd9be155e0c7f057/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:50.060647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee71bf-107d-41b2-b3fd-d176764c26ff None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:50.060941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee71bf-107d-41b2-b3fd-d176764c26ff None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:50.061431 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee71bf-107d-41b2-b3fd-d176764c26ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:50.092451 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-37ee71bf-107d-41b2-b3fd-d176764c26ff None None] MFA Rules not processed for user `256f7cc723aa48fabd9be155e0c7f057`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:50.093763 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-37ee71bf-107d-41b2-b3fd-d176764c26ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:50.172657 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1334/2659] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:50.185378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd29fef-207c-4061-a6bf-4f337b5ac8b2 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:50.187133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-bcd29fef-207c-4061-a6bf-4f337b5ac8b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:50.210862 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:50.253838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd29fef-207c-4061-a6bf-4f337b5ac8b2 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:50.255186 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee71bf-107d-41b2-b3fd-d176764c26ff None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:50.256347 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-37ee71bf-107d-41b2-b3fd-d176764c26ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:50.258638 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf4edc-e3d7-48a6-8c3a-4fb6c0ac9dfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:50.259670 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf4edc-e3d7-48a6-8c3a-4fb6c0ac9dfc service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:50.260142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf4edc-e3d7-48a6-8c3a-4fb6c0ac9dfc service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:50.260606 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf4edc-e3d7-48a6-8c3a-4fb6c0ac9dfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:50.286814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf4edc-e3d7-48a6-8c3a-4fb6c0ac9dfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:50.289050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf4edc-e3d7-48a6-8c3a-4fb6c0ac9dfc service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:50.302596 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1326/2660] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:50.311617 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:50.330119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee71bf-107d-41b2-b3fd-d176764c26ff None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:50.334154 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47540971-b716-4a1c-be77-183530c70334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:50.335323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47540971-b716-4a1c-be77-183530c70334 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:50.335323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47540971-b716-4a1c-be77-183530c70334 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:50.335323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47540971-b716-4a1c-be77-183530c70334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:50.337576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-47540971-b716-4a1c-be77-183530c70334 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:50.404255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47540971-b716-4a1c-be77-183530c70334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:50.405744 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47540971-b716-4a1c-be77-183530c70334 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:50.415432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1335/2661] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:13:50.486839 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf4edc-e3d7-48a6-8c3a-4fb6c0ac9dfc service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:50.497114 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf4edc-e3d7-48a6-8c3a-4fb6c0ac9dfc service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:50.501058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712763cc-2bf6-472f-b1fe-b3de31eb7f0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:50.501731 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712763cc-2bf6-472f-b1fe-b3de31eb7f0b service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:50.501996 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712763cc-2bf6-472f-b1fe-b3de31eb7f0b service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:50.502269 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712763cc-2bf6-472f-b1fe-b3de31eb7f0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:50.560170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97e2fb-bc81-4d9f-a0de-f0ceaa856de0 None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:50.560519 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97e2fb-bc81-4d9f-a0de-f0ceaa856de0 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:50.561077 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97e2fb-bc81-4d9f-a0de-f0ceaa856de0 None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:50.562988 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1336/2662] 10.222.0.94 () {54 vars in 921 bytes} [Tue Jul 26 15:13:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:13:50.569039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad70d8-ed01-4d44-90c6-1f773ec51e63 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:50.569267 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad70d8-ed01-4d44-90c6-1f773ec51e63 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:50.569529 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad70d8-ed01-4d44-90c6-1f773ec51e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:50.578947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712763cc-2bf6-472f-b1fe-b3de31eb7f0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:50.579916 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712763cc-2bf6-472f-b1fe-b3de31eb7f0b service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:50.600809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-ecad70d8-ed01-4d44-90c6-1f773ec51e63 None None] MFA Rules not processed for user `6a4278fe995e432d88f4346d6f54218b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:50.604195 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ecad70d8-ed01-4d44-90c6-1f773ec51e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:50.605223 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1327/2663] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:50.698402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1337/2664] 10.222.0.94 () {58 vars in 1018 bytes} [Tue Jul 26 15:13:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:50.710595 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712763cc-2bf6-472f-b1fe-b3de31eb7f0b service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:50.713016 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-712763cc-2bf6-472f-b1fe-b3de31eb7f0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:50.790633 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712763cc-2bf6-472f-b1fe-b3de31eb7f0b service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:50.796901 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0378fb-4de7-4c51-ae7a-9c6a43c0e8c0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b153935910c142aa80792e111872318a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:50.810088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0378fb-4de7-4c51-ae7a-9c6a43c0e8c0 service glance] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:50.810088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0378fb-4de7-4c51-ae7a-9c6a43c0e8c0 service glance] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:50.810088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0378fb-4de7-4c51-ae7a-9c6a43c0e8c0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:50.812525 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0378fb-4de7-4c51-ae7a-9c6a43c0e8c0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:50.812892 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0378fb-4de7-4c51-ae7a-9c6a43c0e8c0 service glance] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:50.828149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1328/2665] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:51.216067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad70d8-ed01-4d44-90c6-1f773ec51e63 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:51.287947 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad70d8-ed01-4d44-90c6-1f773ec51e63 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:51.293835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd400d52-9b1d-42c2-97c0-cbd2ec47ff77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:51.294584 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd400d52-9b1d-42c2-97c0-cbd2ec47ff77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:51.294928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd400d52-9b1d-42c2-97c0-cbd2ec47ff77 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:51.295274 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd400d52-9b1d-42c2-97c0-cbd2ec47ff77 admin admin] PATH_INFO: `/v3/users/359d68c3c6be4e30817e28cd0be342c8` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:51.296686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd400d52-9b1d-42c2-97c0-cbd2ec47ff77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=359d68c3c6be4e30817e28cd0be342c8)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:51.300375 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd400d52-9b1d-42c2-97c0-cbd2ec47ff77 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:51.325698 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:51.684958 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-dd400d52-9b1d-42c2-97c0-cbd2ec47ff77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '359d68c3c6be4e30817e28cd0be342c8'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:51.696390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-dd400d52-9b1d-42c2-97c0-cbd2ec47ff77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '359d68c3c6be4e30817e28cd0be342c8'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:51.819886 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-dd400d52-9b1d-42c2-97c0-cbd2ec47ff77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '359d68c3c6be4e30817e28cd0be342c8'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:51.886545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-dd400d52-9b1d-42c2-97c0-cbd2ec47ff77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '359d68c3c6be4e30817e28cd0be342c8'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:51.888489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-dd400d52-9b1d-42c2-97c0-cbd2ec47ff77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '359d68c3c6be4e30817e28cd0be342c8'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:51.902793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1338/2666] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:51 2022] DELETE /identity/v3/users/359d68c3c6be4e30817e28cd0be342c8 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:51.907353 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780d813-2d4f-45ad-8ecf-c7dd81468f5b None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:51.908351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780d813-2d4f-45ad-8ecf-c7dd81468f5b None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:51.908828 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780d813-2d4f-45ad-8ecf-c7dd81468f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:51.946763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-0780d813-2d4f-45ad-8ecf-c7dd81468f5b None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:51.948075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0780d813-2d4f-45ad-8ecf-c7dd81468f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:52.031515 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1329/2667] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:52.044572 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd400d52-9b1d-42c2-97c0-cbd2ec47ff77 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:52.046265 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-dd400d52-9b1d-42c2-97c0-cbd2ec47ff77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:52.120695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd400d52-9b1d-42c2-97c0-cbd2ec47ff77 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:52.124688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f2792-3c39-4286-9f12-911930c877a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:52.125390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f2792-3c39-4286-9f12-911930c877a1 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:52.125596 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f2792-3c39-4286-9f12-911930c877a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:52.125898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f2792-3c39-4286-9f12-911930c877a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:52.141066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f2792-3c39-4286-9f12-911930c877a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:52.146684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f2792-3c39-4286-9f12-911930c877a1 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:52.174518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1339/2668] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:52.799399 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780d813-2d4f-45ad-8ecf-c7dd81468f5b None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:52.801221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-0780d813-2d4f-45ad-8ecf-c7dd81468f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:52.810187 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:52.816309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780d813-2d4f-45ad-8ecf-c7dd81468f5b None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:52.824190 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:52.824864 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:52.825145 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:52.825426 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] PATH_INFO: `/v3/projects/039ae51ef6364be380785b7ab1f1bfc1` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:52.826668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=039ae51ef6364be380785b7ab1f1bfc1)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:52.828631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:53.006920 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '039ae51ef6364be380785b7ab1f1bfc1', 'request_id': 'req-b25be493-ffac-4f9f-8418-53001f516441'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:53.017072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '039ae51ef6364be380785b7ab1f1bfc1', 'request_id': 'req-b25be493-ffac-4f9f-8418-53001f516441'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:53.024302 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '039ae51ef6364be380785b7ab1f1bfc1', 'request_id': 'req-b25be493-ffac-4f9f-8418-53001f516441'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:53.141544 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] The token cache is being invalidate because project 039ae51ef6364be380785b7ab1f1bfc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:53.142132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:53.142845 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:53.144706 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1330/2669] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:52 2022] DELETE /identity/v3/projects/039ae51ef6364be380785b7ab1f1bfc1 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:53.148840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f2792-3c39-4286-9f12-911930c877a1 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:53.160229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f2792-3c39-4286-9f12-911930c877a1 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:53.231647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1098d-bbf0-46e0-ab82-91d48d4c1377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:53.232329 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1098d-bbf0-46e0-ab82-91d48d4c1377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:53.232609 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1098d-bbf0-46e0-ab82-91d48d4c1377 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:53.232884 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1098d-bbf0-46e0-ab82-91d48d4c1377 admin admin] PATH_INFO: `/v3/users/256f7cc723aa48fabd9be155e0c7f057` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:53.234118 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1098d-bbf0-46e0-ab82-91d48d4c1377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=256f7cc723aa48fabd9be155e0c7f057)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:53.236758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1098d-bbf0-46e0-ab82-91d48d4c1377 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:53.516466 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:53.518902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-37c1098d-bbf0-46e0-ab82-91d48d4c1377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '256f7cc723aa48fabd9be155e0c7f057'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:53.527791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-37c1098d-bbf0-46e0-ab82-91d48d4c1377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '256f7cc723aa48fabd9be155e0c7f057'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:53.688924 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-37c1098d-bbf0-46e0-ab82-91d48d4c1377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '256f7cc723aa48fabd9be155e0c7f057'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:53.705308 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-37c1098d-bbf0-46e0-ab82-91d48d4c1377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '256f7cc723aa48fabd9be155e0c7f057'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:53.706058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-37c1098d-bbf0-46e0-ab82-91d48d4c1377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '256f7cc723aa48fabd9be155e0c7f057'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:53.713343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1340/2670] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:53 2022] DELETE /identity/v3/users/256f7cc723aa48fabd9be155e0c7f057 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:54.271218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:54.273624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:54.314421 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25be493-ffac-4f9f-8418-53001f516441 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:54.410758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92300597-bf46-4226-8aeb-1dca298e08f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:54.411582 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92300597-bf46-4226-8aeb-1dca298e08f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:54.411872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92300597-bf46-4226-8aeb-1dca298e08f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:54.412150 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92300597-bf46-4226-8aeb-1dca298e08f3 admin admin] PATH_INFO: `/v3/projects/2b25677e9b62477aba9adfc897712a77` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:54.413453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92300597-bf46-4226-8aeb-1dca298e08f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b25677e9b62477aba9adfc897712a77)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:54.416044 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92300597-bf46-4226-8aeb-1dca298e08f3 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:54.594141 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-92300597-bf46-4226-8aeb-1dca298e08f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b25677e9b62477aba9adfc897712a77', 'request_id': 'req-92300597-bf46-4226-8aeb-1dca298e08f3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:54.602169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-92300597-bf46-4226-8aeb-1dca298e08f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b25677e9b62477aba9adfc897712a77', 'request_id': 'req-92300597-bf46-4226-8aeb-1dca298e08f3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:54.609498 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-92300597-bf46-4226-8aeb-1dca298e08f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b25677e9b62477aba9adfc897712a77', 'request_id': 'req-92300597-bf46-4226-8aeb-1dca298e08f3'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:54.778751 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-92300597-bf46-4226-8aeb-1dca298e08f3 admin admin] The token cache is being invalidate because project 2b25677e9b62477aba9adfc897712a77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:54.779397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-92300597-bf46-4226-8aeb-1dca298e08f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:54.780421 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-92300597-bf46-4226-8aeb-1dca298e08f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:54.781996 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1331/2671] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:54 2022] DELETE /identity/v3/projects/2b25677e9b62477aba9adfc897712a77 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:54.786976 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94474e1b-937f-4a75-b689-35faea26d95e None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:54.787805 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94474e1b-937f-4a75-b689-35faea26d95e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:54.788001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94474e1b-937f-4a75-b689-35faea26d95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:54.809947 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-94474e1b-937f-4a75-b689-35faea26d95e None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:54.811518 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-94474e1b-937f-4a75-b689-35faea26d95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:54.872955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1341/2672] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:54.880634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693dede-c748-4375-9fe1-efe138a4f225 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:54.881224 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693dede-c748-4375-9fe1-efe138a4f225 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:54.881466 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693dede-c748-4375-9fe1-efe138a4f225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:54.907394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-9693dede-c748-4375-9fe1-efe138a4f225 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:54.908759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9693dede-c748-4375-9fe1-efe138a4f225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:54.932010 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1332/2673] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:54.937514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94474e1b-937f-4a75-b689-35faea26d95e None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:54.950728 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94474e1b-937f-4a75-b689-35faea26d95e None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:54.956915 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3ce4f-afee-4c21-bd01-5700984927b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:54.957451 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3ce4f-afee-4c21-bd01-5700984927b0 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:54.957840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3ce4f-afee-4c21-bd01-5700984927b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:54.958217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3ce4f-afee-4c21-bd01-5700984927b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:54.959039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3ce4f-afee-4c21-bd01-5700984927b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:13:54.959499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3ce4f-afee-4c21-bd01-5700984927b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:54.961689 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3ce4f-afee-4c21-bd01-5700984927b0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:54.977259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1342/2674] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:13:54.982496 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693dede-c748-4375-9fe1-efe138a4f225 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:55.012443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693dede-c748-4375-9fe1-efe138a4f225 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:55.012443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ed35d-a7f6-4f8a-90a2-e6072d0f2eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:55.012443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ed35d-a7f6-4f8a-90a2-e6072d0f2eaf admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:55.012443 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ed35d-a7f6-4f8a-90a2-e6072d0f2eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:55.013802 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ed35d-a7f6-4f8a-90a2-e6072d0f2eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:55.013802 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ed35d-a7f6-4f8a-90a2-e6072d0f2eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:55.013802 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ed35d-a7f6-4f8a-90a2-e6072d0f2eaf admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:55.016036 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:55.133596 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1333/2675] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:54 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:13:55.138513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3ce4f-afee-4c21-bd01-5700984927b0 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:55.155080 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3ce4f-afee-4c21-bd01-5700984927b0 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:55.225139 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162370-76e5-4060-9902-faa8f7b1ac39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:55.225902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162370-76e5-4060-9902-faa8f7b1ac39 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:55.226270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162370-76e5-4060-9902-faa8f7b1ac39 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:55.226619 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e162370-76e5-4060-9902-faa8f7b1ac39 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:55.227364 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e162370-76e5-4060-9902-faa8f7b1ac39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:55.230362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e162370-76e5-4060-9902-faa8f7b1ac39 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:55.238801 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ed35d-a7f6-4f8a-90a2-e6072d0f2eaf admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:55.241148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e89ed35d-a7f6-4f8a-90a2-e6072d0f2eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:55.333127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ed35d-a7f6-4f8a-90a2-e6072d0f2eaf admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:55.340276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11087905-5f62-4eab-a442-f32f7fc5b1d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a0728e2f6249bdb45d6f44fb15aa4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:55.341516 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11087905-5f62-4eab-a442-f32f7fc5b1d7 service placement] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:55.341872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11087905-5f62-4eab-a442-f32f7fc5b1d7 service placement] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:55.342416 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11087905-5f62-4eab-a442-f32f7fc5b1d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:55.344924 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-11087905-5f62-4eab-a442-f32f7fc5b1d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:55.349238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1343/2676] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:55 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:55.354361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162370-76e5-4060-9902-faa8f7b1ac39 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:55.365413 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11087905-5f62-4eab-a442-f32f7fc5b1d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:55.365731 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e162370-76e5-4060-9902-faa8f7b1ac39 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:55.368296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11087905-5f62-4eab-a442-f32f7fc5b1d7 service placement] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:55.371703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903642c-dfa4-4c7a-a7f6-86dc15d28eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:55.374964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903642c-dfa4-4c7a-a7f6-86dc15d28eaa admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:55.374964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903642c-dfa4-4c7a-a7f6-86dc15d28eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:55.374964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903642c-dfa4-4c7a-a7f6-86dc15d28eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:55.374964 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903642c-dfa4-4c7a-a7f6-86dc15d28eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:55.376435 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903642c-dfa4-4c7a-a7f6-86dc15d28eaa admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:55.383555 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1334/2677] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:13:55.393913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1344/2678] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:13:55.402613 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11087905-5f62-4eab-a442-f32f7fc5b1d7 service placement] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:55.416758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11087905-5f62-4eab-a442-f32f7fc5b1d7 service placement] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:55.423493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4fc42-71ce-44f7-9d00-c872972ae93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:55.424531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4fc42-71ce-44f7-9d00-c872972ae93b admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:55.424968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4fc42-71ce-44f7-9d00-c872972ae93b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:55.425411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4fc42-71ce-44f7-9d00-c872972ae93b admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:55.426583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4fc42-71ce-44f7-9d00-c872972ae93b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:55.429022 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4fc42-71ce-44f7-9d00-c872972ae93b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:55.445144 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1335/2679] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:55.452942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903642c-dfa4-4c7a-a7f6-86dc15d28eaa admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:55.468192 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903642c-dfa4-4c7a-a7f6-86dc15d28eaa admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:55.476075 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25295a6c-b6ea-489b-b242-94f6b5436c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:55.476975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25295a6c-b6ea-489b-b242-94f6b5436c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:55.477258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25295a6c-b6ea-489b-b242-94f6b5436c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:55.477536 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25295a6c-b6ea-489b-b242-94f6b5436c96 admin admin] PATH_INFO: `/v3/projects/0b69ff3fcc5842eeac935ebf0d40d5df/users/3cc75e09b17942838095326534f5617e/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:55.500986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25295a6c-b6ea-489b-b242-94f6b5436c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b69ff3fcc5842eeac935ebf0d40d5df, user_id=3cc75e09b17942838095326534f5617e, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:55.503663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25295a6c-b6ea-489b-b242-94f6b5436c96 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:55.609905 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1345/2680] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:55 2022] PUT /identity/v3/projects/0b69ff3fcc5842eeac935ebf0d40d5df/users/3cc75e09b17942838095326534f5617e/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:55.688615 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc53ac4-5a34-49b1-ac64-f27f23812c4f None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:55.688933 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc53ac4-5a34-49b1-ac64-f27f23812c4f None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:55.689514 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc53ac4-5a34-49b1-ac64-f27f23812c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:55.721456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-cdc53ac4-5a34-49b1-ac64-f27f23812c4f None None] MFA Rules not processed for user `3cc75e09b17942838095326534f5617e`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:55.722826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-cdc53ac4-5a34-49b1-ac64-f27f23812c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:55.815317 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1336/2681] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:55.827543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25295a6c-b6ea-489b-b242-94f6b5436c96 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:55.830245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-25295a6c-b6ea-489b-b242-94f6b5436c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:55.837837 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:55.901797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25295a6c-b6ea-489b-b242-94f6b5436c96 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:55.911822 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b0bf9-64d9-47a3-b326-c807713576ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:55.916265 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b0bf9-64d9-47a3-b326-c807713576ec service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:55.917334 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b0bf9-64d9-47a3-b326-c807713576ec service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:55.917683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b0bf9-64d9-47a3-b326-c807713576ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:55.937990 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b0bf9-64d9-47a3-b326-c807713576ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:55.939898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b0bf9-64d9-47a3-b326-c807713576ec service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:55.960520 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1346/2682] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:13:55.973119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc53ac4-5a34-49b1-ac64-f27f23812c4f None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:56.057432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc53ac4-5a34-49b1-ac64-f27f23812c4f None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:56.065747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2e250-3b9a-4d39-bd6b-33db97706739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:56.066417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2e250-3b9a-4d39-bd6b-33db97706739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:56.066691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2e250-3b9a-4d39-bd6b-33db97706739 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:56.066968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2e250-3b9a-4d39-bd6b-33db97706739 admin admin] PATH_INFO: `/v3/users/3cc75e09b17942838095326534f5617e` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:56.068766 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2e250-3b9a-4d39-bd6b-33db97706739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc75e09b17942838095326534f5617e)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:56.070688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2e250-3b9a-4d39-bd6b-33db97706739 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:56.238004 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:56.321975 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b0bf9-64d9-47a3-b326-c807713576ec service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:56.323822 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2a2b0bf9-64d9-47a3-b326-c807713576ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:56.339983 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b0bf9-64d9-47a3-b326-c807713576ec service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:56.345392 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd2257-e6e6-4149-a652-55b9b577ba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:56.346047 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd2257-e6e6-4149-a652-55b9b577ba20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:56.346331 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd2257-e6e6-4149-a652-55b9b577ba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:56.346602 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd2257-e6e6-4149-a652-55b9b577ba20 admin admin] PATH_INFO: `/v3/users/3009e4514812421a87e7e604d2326556` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:56.348013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd2257-e6e6-4149-a652-55b9b577ba20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3009e4514812421a87e7e604d2326556)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:56.350587 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd2257-e6e6-4149-a652-55b9b577ba20 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:56.364852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1ee2e250-3b9a-4d39-bd6b-33db97706739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc75e09b17942838095326534f5617e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:56.365531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1ee2e250-3b9a-4d39-bd6b-33db97706739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc75e09b17942838095326534f5617e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:56.366430 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1ee2e250-3b9a-4d39-bd6b-33db97706739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc75e09b17942838095326534f5617e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:56.374230 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1ee2e250-3b9a-4d39-bd6b-33db97706739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc75e09b17942838095326534f5617e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:56.387243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-1ee2e250-3b9a-4d39-bd6b-33db97706739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc75e09b17942838095326534f5617e'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:56.542713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1337/2683] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:55 2022] DELETE /identity/v3/users/3cc75e09b17942838095326534f5617e => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:56.547456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fa8ed-f482-4588-b5e6-4fba566cd90f None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:56.547772 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fa8ed-f482-4588-b5e6-4fba566cd90f None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:56.548071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fa8ed-f482-4588-b5e6-4fba566cd90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:56.569100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-cb1fa8ed-f482-4588-b5e6-4fba566cd90f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:56.570362 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-cb1fa8ed-f482-4588-b5e6-4fba566cd90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:56.632899 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1338/2684] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:13:56.642658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fa8ed-f482-4588-b5e6-4fba566cd90f None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:56.644136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-cb1fa8ed-f482-4588-b5e6-4fba566cd90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:56.689753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1cdd2257-e6e6-4149-a652-55b9b577ba20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3009e4514812421a87e7e604d2326556'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:56.698210 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1cdd2257-e6e6-4149-a652-55b9b577ba20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3009e4514812421a87e7e604d2326556'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:56.714991 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fa8ed-f482-4588-b5e6-4fba566cd90f None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:56.719986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc550811-c767-4976-8bc9-d08890ca0790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:56.720626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc550811-c767-4976-8bc9-d08890ca0790 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:56.720962 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc550811-c767-4976-8bc9-d08890ca0790 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:56.721243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc550811-c767-4976-8bc9-d08890ca0790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:56.733123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc550811-c767-4976-8bc9-d08890ca0790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:56.735073 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc550811-c767-4976-8bc9-d08890ca0790 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:56.745946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1339/2685] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:13:56.786385 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1cdd2257-e6e6-4149-a652-55b9b577ba20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3009e4514812421a87e7e604d2326556'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:56.802352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1cdd2257-e6e6-4149-a652-55b9b577ba20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3009e4514812421a87e7e604d2326556'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:56.803564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-1cdd2257-e6e6-4149-a652-55b9b577ba20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3009e4514812421a87e7e604d2326556'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:56.812269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1347/2686] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:56 2022] DELETE /identity/v3/users/3009e4514812421a87e7e604d2326556 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:57.188992 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc550811-c767-4976-8bc9-d08890ca0790 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:57.191573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-cc550811-c767-4976-8bc9-d08890ca0790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:57.214626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc550811-c767-4976-8bc9-d08890ca0790 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:57.236004 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:57.320225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:57.320965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:57.321291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:57.322310 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] PATH_INFO: `/v3/projects/912b783c262149598bbc2b398aec7ab4` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:57.324093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] RBAC: Authorizing `identity:delete_project(project_id=912b783c262149598bbc2b398aec7ab4)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:57.326165 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:57.356894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd2257-e6e6-4149-a652-55b9b577ba20 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:57.363912 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1cdd2257-e6e6-4149-a652-55b9b577ba20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:57.425207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd2257-e6e6-4149-a652-55b9b577ba20 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:57.433118 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:57.433970 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:57.434258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:57.434535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] PATH_INFO: `/v3/projects/0b69ff3fcc5842eeac935ebf0d40d5df` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:57.435845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b69ff3fcc5842eeac935ebf0d40d5df)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:57.437783 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:57.476399 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '912b783c262149598bbc2b398aec7ab4', 'request_id': 'req-8e317c04-f1b7-408a-80f3-46259885109b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:57.484248 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '912b783c262149598bbc2b398aec7ab4', 'request_id': 'req-8e317c04-f1b7-408a-80f3-46259885109b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:57.490697 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '912b783c262149598bbc2b398aec7ab4', 'request_id': 'req-8e317c04-f1b7-408a-80f3-46259885109b'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:57.787245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] The token cache is being invalidate because project 912b783c262149598bbc2b398aec7ab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:57.787245 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:57.799852 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:57.803485 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1340/2687] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:57 2022] DELETE /identity/v3/projects/912b783c262149598bbc2b398aec7ab4 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:57.810404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b69ff3fcc5842eeac935ebf0d40d5df', 'request_id': 'req-6bbabc5b-b5e7-416c-b243-04f25795f597'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:57.813235 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:57.814695 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:57.830231 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b69ff3fcc5842eeac935ebf0d40d5df', 'request_id': 'req-6bbabc5b-b5e7-416c-b243-04f25795f597'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:57.894739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e317c04-f1b7-408a-80f3-46259885109b admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:57.965768 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b69ff3fcc5842eeac935ebf0d40d5df', 'request_id': 'req-6bbabc5b-b5e7-416c-b243-04f25795f597'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:57.972448 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229532ff-af0d-4ac8-94ce-275ca5fa3c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:57.973784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229532ff-af0d-4ac8-94ce-275ca5fa3c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:57.974239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229532ff-af0d-4ac8-94ce-275ca5fa3c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:57.974598 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229532ff-af0d-4ac8-94ce-275ca5fa3c0d admin admin] PATH_INFO: `/v3/users/b5a1e90bb5b940c0a8340e1fda48a12a` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:57.976006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229532ff-af0d-4ac8-94ce-275ca5fa3c0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5a1e90bb5b940c0a8340e1fda48a12a)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:57.978814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229532ff-af0d-4ac8-94ce-275ca5fa3c0d admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:57.981754 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] The token cache is being invalidate because project 0b69ff3fcc5842eeac935ebf0d40d5df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:57.982051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:57.982724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:57.984961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1348/2688] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:57 2022] DELETE /identity/v3/projects/0b69ff3fcc5842eeac935ebf0d40d5df => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:57.986130 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:57.991238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:58.000968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:58.057953 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbabc5b-b5e7-416c-b243-04f25795f597 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:58.061631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58eabb7-e24a-4ea8-b668-3ffb701dea41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:58.062247 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58eabb7-e24a-4ea8-b668-3ffb701dea41 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:58.062524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58eabb7-e24a-4ea8-b668-3ffb701dea41 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:58.062801 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58eabb7-e24a-4ea8-b668-3ffb701dea41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:58.064658 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c58eabb7-e24a-4ea8-b668-3ffb701dea41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:58.076802 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58eabb7-e24a-4ea8-b668-3ffb701dea41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:58.078372 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58eabb7-e24a-4ea8-b668-3ffb701dea41 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:58.088820 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1349/2689] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:13:58.090185 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9f870-02a8-4156-baf2-66fdfed48188 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:58.090477 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9f870-02a8-4156-baf2-66fdfed48188 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:58.090762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9f870-02a8-4156-baf2-66fdfed48188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:58.112409 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-42f9f870-02a8-4156-baf2-66fdfed48188 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:58.113623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-42f9f870-02a8-4156-baf2-66fdfed48188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:58.127898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1350/2690] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:58.130116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbd3a5-49d9-4873-9ee7-7aacfc1d0a8f None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:58.130578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbd3a5-49d9-4873-9ee7-7aacfc1d0a8f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:58.131026 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbd3a5-49d9-4873-9ee7-7aacfc1d0a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:58.154939 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-98cbd3a5-49d9-4873-9ee7-7aacfc1d0a8f None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:58.156187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-98cbd3a5-49d9-4873-9ee7-7aacfc1d0a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:58.179873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-229532ff-af0d-4ac8-94ce-275ca5fa3c0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5a1e90bb5b940c0a8340e1fda48a12a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:58.182315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-229532ff-af0d-4ac8-94ce-275ca5fa3c0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5a1e90bb5b940c0a8340e1fda48a12a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:58.183430 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-229532ff-af0d-4ac8-94ce-275ca5fa3c0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5a1e90bb5b940c0a8340e1fda48a12a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:58.195109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-229532ff-af0d-4ac8-94ce-275ca5fa3c0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5a1e90bb5b940c0a8340e1fda48a12a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:58.212566 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-229532ff-af0d-4ac8-94ce-275ca5fa3c0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5a1e90bb5b940c0a8340e1fda48a12a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:58.226623 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1351/2691] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 15:13:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:58.227027 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:58.229177 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91157d34-046a-4663-9b8d-c9344ec107a5 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:58.229563 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91157d34-046a-4663-9b8d-c9344ec107a5 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:58.236320 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91157d34-046a-4663-9b8d-c9344ec107a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:58.252916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-91157d34-046a-4663-9b8d-c9344ec107a5 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:58.254202 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-91157d34-046a-4663-9b8d-c9344ec107a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:58.262745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1352/2692] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:58.264683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91157d34-046a-4663-9b8d-c9344ec107a5 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:58.266807 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-91157d34-046a-4663-9b8d-c9344ec107a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:58.292974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1341/2693] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:13:57 2022] DELETE /identity/v3/users/b5a1e90bb5b940c0a8340e1fda48a12a => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:13:58.292974 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229532ff-af0d-4ac8-94ce-275ca5fa3c0d admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:58.394410 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229532ff-af0d-4ac8-94ce-275ca5fa3c0d admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:58.401927 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe689ed7-0f53-4d04-b0dd-909808200a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:58.402845 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe689ed7-0f53-4d04-b0dd-909808200a1c admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:58.403336 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe689ed7-0f53-4d04-b0dd-909808200a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:58.403824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe689ed7-0f53-4d04-b0dd-909808200a1c admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:58.404768 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe689ed7-0f53-4d04-b0dd-909808200a1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:13:58.405256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe689ed7-0f53-4d04-b0dd-909808200a1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:58.407921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe689ed7-0f53-4d04-b0dd-909808200a1c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:58.422739 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91157d34-046a-4663-9b8d-c9344ec107a5 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:58.426426 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1342/2694] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:13:58.438231 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe689ed7-0f53-4d04-b0dd-909808200a1c admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:58.456013 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe689ed7-0f53-4d04-b0dd-909808200a1c admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:58.462847 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ff015-1312-4a80-8833-cc86777d5899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:58.463220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ff015-1312-4a80-8833-cc86777d5899 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:58.464302 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ff015-1312-4a80-8833-cc86777d5899 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:58.464946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ff015-1312-4a80-8833-cc86777d5899 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:58.466109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ff015-1312-4a80-8833-cc86777d5899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:58.469269 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ff015-1312-4a80-8833-cc86777d5899 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:58.557063 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6d27e-ca5c-4032-a5a8-adfdeaef0d26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:58.557782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6d27e-ca5c-4032-a5a8-adfdeaef0d26 service swift] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:58.558065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6d27e-ca5c-4032-a5a8-adfdeaef0d26 service swift] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:58.558336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6d27e-ca5c-4032-a5a8-adfdeaef0d26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:58.626044 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6d27e-ca5c-4032-a5a8-adfdeaef0d26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:58.627628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6d27e-ca5c-4032-a5a8-adfdeaef0d26 service swift] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:58.639994 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1343/2695] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:58 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:58.641303 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ff015-1312-4a80-8833-cc86777d5899 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:58.643717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1353/2696] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:13:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:13:58.724881 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ff015-1312-4a80-8833-cc86777d5899 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:58.731156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0be5a2-5181-4ff4-a670-cb8c6652e125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:58.731830 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0be5a2-5181-4ff4-a670-cb8c6652e125 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:58.732109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0be5a2-5181-4ff4-a670-cb8c6652e125 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:58.732419 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0be5a2-5181-4ff4-a670-cb8c6652e125 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:58.733105 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0be5a2-5181-4ff4-a670-cb8c6652e125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:58.735072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0be5a2-5181-4ff4-a670-cb8c6652e125 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:58.755395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6d27e-ca5c-4032-a5a8-adfdeaef0d26 service swift] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:58.767801 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d7b6d27e-ca5c-4032-a5a8-adfdeaef0d26 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:58.794933 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6d27e-ca5c-4032-a5a8-adfdeaef0d26 service swift] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:58.800752 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76090f9c-1b67-4900-970f-31b92d2addfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:58.801480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76090f9c-1b67-4900-970f-31b92d2addfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:58.807840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76090f9c-1b67-4900-970f-31b92d2addfc admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:58.807840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76090f9c-1b67-4900-970f-31b92d2addfc admin admin] PATH_INFO: `/v3/projects/f133d4db3602405dbd1717f77ac9d09c` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:58.807840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76090f9c-1b67-4900-970f-31b92d2addfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f133d4db3602405dbd1717f77ac9d09c)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:58.807840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76090f9c-1b67-4900-970f-31b92d2addfc admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:58.974000 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1344/2697] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:58 2022] POST /identity/v3/users => generated 293 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:58.985200 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0be5a2-5181-4ff4-a670-cb8c6652e125 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:58.989349 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0be5a2-5181-4ff4-a670-cb8c6652e125 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:58.997298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5fea8-3488-45a7-9cc6-2978b72e2050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:58.997938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5fea8-3488-45a7-9cc6-2978b72e2050 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:58.998510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5fea8-3488-45a7-9cc6-2978b72e2050 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:58.998842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5fea8-3488-45a7-9cc6-2978b72e2050 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:58.999635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5fea8-3488-45a7-9cc6-2978b72e2050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:59.001793 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5fea8-3488-45a7-9cc6-2978b72e2050 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:59.014573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1345/2698] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:13:59.019090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5fea8-3488-45a7-9cc6-2978b72e2050 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:59.048543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-76090f9c-1b67-4900-970f-31b92d2addfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f133d4db3602405dbd1717f77ac9d09c', 'request_id': 'req-76090f9c-1b67-4900-970f-31b92d2addfc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:59.057732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5fea8-3488-45a7-9cc6-2978b72e2050 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:59.065749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-76090f9c-1b67-4900-970f-31b92d2addfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f133d4db3602405dbd1717f77ac9d09c', 'request_id': 'req-76090f9c-1b67-4900-970f-31b92d2addfc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:59.183380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68379ecd-9814-4afc-9933-9c7a691bbfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:59.184042 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68379ecd-9814-4afc-9933-9c7a691bbfdb admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:59.184667 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68379ecd-9814-4afc-9933-9c7a691bbfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:59.185155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68379ecd-9814-4afc-9933-9c7a691bbfdb admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:59.186534 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68379ecd-9814-4afc-9933-9c7a691bbfdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:59.187625 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68379ecd-9814-4afc-9933-9c7a691bbfdb admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:59.203135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-76090f9c-1b67-4900-970f-31b92d2addfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f133d4db3602405dbd1717f77ac9d09c', 'request_id': 'req-76090f9c-1b67-4900-970f-31b92d2addfc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:59.216194 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1346/2699] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:13:59.220538 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68379ecd-9814-4afc-9933-9c7a691bbfdb admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:59.223327 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-76090f9c-1b67-4900-970f-31b92d2addfc admin admin] The token cache is being invalidate because project f133d4db3602405dbd1717f77ac9d09c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:13:59.223782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-76090f9c-1b67-4900-970f-31b92d2addfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:59.224606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-76090f9c-1b67-4900-970f-31b92d2addfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:13:59.226547 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1354/2700] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:13:58 2022] DELETE /identity/v3/projects/f133d4db3602405dbd1717f77ac9d09c => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:59.234782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107dd2b-b462-468d-8437-8a64706f5e55 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:59.235544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107dd2b-b462-468d-8437-8a64706f5e55 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:59.235544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107dd2b-b462-468d-8437-8a64706f5e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:59.260716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68379ecd-9814-4afc-9933-9c7a691bbfdb admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:59.272860 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae459c5-9b97-411b-abbf-1310c26b7462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:59.274764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae459c5-9b97-411b-abbf-1310c26b7462 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:59.274764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae459c5-9b97-411b-abbf-1310c26b7462 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:59.274971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-2107dd2b-b462-468d-8437-8a64706f5e55 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:59.275257 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae459c5-9b97-411b-abbf-1310c26b7462 admin admin] PATH_INFO: `/v3/projects/76685d86ee9d408e9cc57e99f5896307/users/54f8741cfc7a4953b53470368207edfc/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:59.275863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-2107dd2b-b462-468d-8437-8a64706f5e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:59.297668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1355/2701] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:59.300031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23bec1-daf9-41e1-a079-54e52ddb6fb9 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:59.300631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23bec1-daf9-41e1-a079-54e52ddb6fb9 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:59.301211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23bec1-daf9-41e1-a079-54e52ddb6fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:59.330426 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae459c5-9b97-411b-abbf-1310c26b7462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76685d86ee9d408e9cc57e99f5896307, user_id=54f8741cfc7a4953b53470368207edfc, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:59.333656 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae459c5-9b97-411b-abbf-1310c26b7462 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:59.338606 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-0d23bec1-daf9-41e1-a079-54e52ddb6fb9 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:59.340900 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-0d23bec1-daf9-41e1-a079-54e52ddb6fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:59.351184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1356/2702] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:59.356710 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23bec1-daf9-41e1-a079-54e52ddb6fb9 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:59.368889 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23bec1-daf9-41e1-a079-54e52ddb6fb9 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:59.374523 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b5ecc-29fc-4a77-b4cf-dfada7525f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:59.375290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b5ecc-29fc-4a77-b4cf-dfada7525f36 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:59.375695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b5ecc-29fc-4a77-b4cf-dfada7525f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:59.376034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b5ecc-29fc-4a77-b4cf-dfada7525f36 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:59.376968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b5ecc-29fc-4a77-b4cf-dfada7525f36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:13:59.377372 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b5ecc-29fc-4a77-b4cf-dfada7525f36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:59.379324 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b5ecc-29fc-4a77-b4cf-dfada7525f36 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:59.394203 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1357/2703] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:13:59.399448 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b5ecc-29fc-4a77-b4cf-dfada7525f36 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:59.412580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b5ecc-29fc-4a77-b4cf-dfada7525f36 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:59.419489 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bac6f5-41c1-4c80-81b9-b69321b7d39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:59.420330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bac6f5-41c1-4c80-81b9-b69321b7d39a admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:59.420681 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bac6f5-41c1-4c80-81b9-b69321b7d39a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:59.421061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bac6f5-41c1-4c80-81b9-b69321b7d39a admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:59.421760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bac6f5-41c1-4c80-81b9-b69321b7d39a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:59.423815 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bac6f5-41c1-4c80-81b9-b69321b7d39a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:59.483824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1347/2704] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:13:59 2022] PUT /identity/v3/projects/76685d86ee9d408e9cc57e99f5896307/users/54f8741cfc7a4953b53470368207edfc/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:13:59.496548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b268ab2-283c-44b5-8956-d5d874a38e29 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:59.496827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b268ab2-283c-44b5-8956-d5d874a38e29 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:59.497093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b268ab2-283c-44b5-8956-d5d874a38e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:59.549675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-7b268ab2-283c-44b5-8956-d5d874a38e29 None None] MFA Rules not processed for user `54f8741cfc7a4953b53470368207edfc`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:13:59.551325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-7b268ab2-283c-44b5-8956-d5d874a38e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:59.608634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1358/2705] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:13:59 2022] POST /identity/v3/projects => generated 377 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:13:59.613323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bac6f5-41c1-4c80-81b9-b69321b7d39a admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:59.760978 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1348/2706] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:13:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:13:59.769051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:59.772719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b268ab2-283c-44b5-8956-d5d874a38e29 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:13:59.775960 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-7b268ab2-283c-44b5-8956-d5d874a38e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:13:59.844978 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bac6f5-41c1-4c80-81b9-b69321b7d39a admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:13:59.867225 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4b28c-0d11-4ba6-b45b-0890e70412b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:13:59.870067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4b28c-0d11-4ba6-b45b-0890e70412b5 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:13:59.871718 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4b28c-0d11-4ba6-b45b-0890e70412b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:13:59.871979 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4b28c-0d11-4ba6-b45b-0890e70412b5 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:13:59.873028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4b28c-0d11-4ba6-b45b-0890e70412b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:13:59.878098 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4b28c-0d11-4ba6-b45b-0890e70412b5 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:13:59.890086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:59.915839 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:13:59.961680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b268ab2-283c-44b5-8956-d5d874a38e29 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:00.033749 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352441ef-436f-42d5-98ba-bee4f5cd97d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:00.034680 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352441ef-436f-42d5-98ba-bee4f5cd97d6 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:00.035025 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352441ef-436f-42d5-98ba-bee4f5cd97d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:00.035864 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352441ef-436f-42d5-98ba-bee4f5cd97d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:00.080057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1359/2707] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:13:59 2022] POST /identity/v3/users => generated 289 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:14:00.084048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4b28c-0d11-4ba6-b45b-0890e70412b5 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:00.119390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4b28c-0d11-4ba6-b45b-0890e70412b5 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:00.131155 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e80e08-d40d-4459-8a77-83336a525d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:00.131966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e80e08-d40d-4459-8a77-83336a525d12 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:00.132373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e80e08-d40d-4459-8a77-83336a525d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:00.132851 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e80e08-d40d-4459-8a77-83336a525d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:00.133599 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e80e08-d40d-4459-8a77-83336a525d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:00.139575 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e80e08-d40d-4459-8a77-83336a525d12 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:00.162551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1360/2708] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:14:00.167624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352441ef-436f-42d5-98ba-bee4f5cd97d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:00.169895 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352441ef-436f-42d5-98ba-bee4f5cd97d6 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:00.170424 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e80e08-d40d-4459-8a77-83336a525d12 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:00.190033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e80e08-d40d-4459-8a77-83336a525d12 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:00.204694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1349/2709] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:13:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:14:00.221789 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f5064-f403-4679-bddc-6b52df8d2ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:00.222456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f5064-f403-4679-bddc-6b52df8d2ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:00.222835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f5064-f403-4679-bddc-6b52df8d2ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:00.223222 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f5064-f403-4679-bddc-6b52df8d2ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:00.224338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f5064-f403-4679-bddc-6b52df8d2ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:00.230539 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f5064-f403-4679-bddc-6b52df8d2ee7 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:00.275761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352441ef-436f-42d5-98ba-bee4f5cd97d6 service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:00.276564 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1361/2710] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:14:00.293186 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352441ef-436f-42d5-98ba-bee4f5cd97d6 service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:00.304070 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e3a5c-670a-43aa-a6d8-fc24f1478531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:00.305110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e3a5c-670a-43aa-a6d8-fc24f1478531 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:00.306369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e3a5c-670a-43aa-a6d8-fc24f1478531 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:00.308466 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e3a5c-670a-43aa-a6d8-fc24f1478531 admin admin] PATH_INFO: `/v3/projects/aba3d424c6e64b9fb2b0c5e16eb42763/users/c11f43d5614d46cd996789819075520d/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:00.346866 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:00.350156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e3a5c-670a-43aa-a6d8-fc24f1478531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba3d424c6e64b9fb2b0c5e16eb42763, user_id=c11f43d5614d46cd996789819075520d, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:00.354086 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e3a5c-670a-43aa-a6d8-fc24f1478531 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:00.486195 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bab10-7066-4882-afcf-a1b8bd78eb63 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:00.487451 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bab10-7066-4882-afcf-a1b8bd78eb63 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:00.488238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bab10-7066-4882-afcf-a1b8bd78eb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:00.492305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1350/2711] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:14:00 2022] PUT /identity/v3/projects/aba3d424c6e64b9fb2b0c5e16eb42763/users/c11f43d5614d46cd996789819075520d/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:14:00.514968 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-035bab10-7066-4882-afcf-a1b8bd78eb63 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:14:00.516758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-035bab10-7066-4882-afcf-a1b8bd78eb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:00.645128 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1362/2712] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:14:00.658823 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e3a5c-670a-43aa-a6d8-fc24f1478531 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:00.660983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-c74e3a5c-670a-43aa-a6d8-fc24f1478531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:00.785036 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e3a5c-670a-43aa-a6d8-fc24f1478531 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:00.792328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58e838-a9b5-4a89-818e-f7981d50a667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:00.793405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58e838-a9b5-4a89-818e-f7981d50a667 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:00.793781 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58e838-a9b5-4a89-818e-f7981d50a667 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:00.794152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58e838-a9b5-4a89-818e-f7981d50a667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:00.813777 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58e838-a9b5-4a89-818e-f7981d50a667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:00.820509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58e838-a9b5-4a89-818e-f7981d50a667 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:00.834830 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1351/2713] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:14:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:14:10.624525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:10.789972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902dec3-9c26-4c4e-af8a-67ae716f8d61 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:10.790566 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902dec3-9c26-4c4e-af8a-67ae716f8d61 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:10.791058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902dec3-9c26-4c4e-af8a-67ae716f8d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:10.828482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:10.829047 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-f902dec3-9c26-4c4e-af8a-67ae716f8d61 None None] MFA Rules not processed for user `c11f43d5614d46cd996789819075520d`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:14:10.830144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f902dec3-9c26-4c4e-af8a-67ae716f8d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:10.916545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1363/2714] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:14:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:14:10.921072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58e838-a9b5-4a89-818e-f7981d50a667 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:10.931979 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58e838-a9b5-4a89-818e-f7981d50a667 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:10.936759 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740c22f-e8c3-4080-9ea5-adeda0c3a437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:10.937402 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740c22f-e8c3-4080-9ea5-adeda0c3a437 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:10.937692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740c22f-e8c3-4080-9ea5-adeda0c3a437 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:10.937966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740c22f-e8c3-4080-9ea5-adeda0c3a437 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:10.938655 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740c22f-e8c3-4080-9ea5-adeda0c3a437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:10.940638 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740c22f-e8c3-4080-9ea5-adeda0c3a437 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:11.053238 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1352/2715] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:14:10 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:14:11.060329 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902dec3-9c26-4c4e-af8a-67ae716f8d61 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:11.077269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902dec3-9c26-4c4e-af8a-67ae716f8d61 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:11.157219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5f127-7d66-4322-9780-9824eb45ef50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:11.158075 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5f127-7d66-4322-9780-9824eb45ef50 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:11.158478 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5f127-7d66-4322-9780-9824eb45ef50 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:11.158863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d5f127-7d66-4322-9780-9824eb45ef50 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:11.159758 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5f127-7d66-4322-9780-9824eb45ef50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:11.161927 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d5f127-7d66-4322-9780-9824eb45ef50 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:11.259241 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1364/2716] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:14:11 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:14:11.263720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740c22f-e8c3-4080-9ea5-adeda0c3a437 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:11.281129 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740c22f-e8c3-4080-9ea5-adeda0c3a437 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:11.285973 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c141a-b216-4535-9054-6c334247968b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:11.286612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c141a-b216-4535-9054-6c334247968b admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:11.286910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c141a-b216-4535-9054-6c334247968b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:11.287169 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c141a-b216-4535-9054-6c334247968b admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:11.288038 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c141a-b216-4535-9054-6c334247968b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:11.290477 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c141a-b216-4535-9054-6c334247968b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:11.302285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1353/2717] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:14:11.307684 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5f127-7d66-4322-9780-9824eb45ef50 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:11.318544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d5f127-7d66-4322-9780-9824eb45ef50 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:11.324111 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a0fba-100e-4409-aca6-1c5cce9f60d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:11.324804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a0fba-100e-4409-aca6-1c5cce9f60d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:11.325087 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a0fba-100e-4409-aca6-1c5cce9f60d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:11.325366 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a0fba-100e-4409-aca6-1c5cce9f60d8 admin admin] PATH_INFO: `/v3/projects/1e0ee14bac364a66b46a76f74172c6ad/users/bd65b0c1f44c41db8f25fbc9848a9173/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:11.342772 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a0fba-100e-4409-aca6-1c5cce9f60d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e0ee14bac364a66b46a76f74172c6ad, user_id=bd65b0c1f44c41db8f25fbc9848a9173, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:11.346048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a0fba-100e-4409-aca6-1c5cce9f60d8 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:11.463546 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1365/2718] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:14:11 2022] PUT /identity/v3/projects/1e0ee14bac364a66b46a76f74172c6ad/users/bd65b0c1f44c41db8f25fbc9848a9173/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:14:11.467675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c141a-b216-4535-9054-6c334247968b admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:11.481236 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c141a-b216-4535-9054-6c334247968b admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:11.560290 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a91c2c-e3ca-4a30-b020-162fd1181ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:11.560917 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a91c2c-e3ca-4a30-b020-162fd1181ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:11.561221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a91c2c-e3ca-4a30-b020-162fd1181ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:11.561505 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a91c2c-e3ca-4a30-b020-162fd1181ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:11.562211 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a91c2c-e3ca-4a30-b020-162fd1181ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:11.563922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a91c2c-e3ca-4a30-b020-162fd1181ff0 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:11.575673 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1354/2719] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:14:11.580153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a0fba-100e-4409-aca6-1c5cce9f60d8 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:11.591874 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a0fba-100e-4409-aca6-1c5cce9f60d8 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:11.596913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bfaab-094b-4ce5-a825-d5b3d3b18948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:11.597695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bfaab-094b-4ce5-a825-d5b3d3b18948 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:11.597981 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bfaab-094b-4ce5-a825-d5b3d3b18948 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:11.598253 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bfaab-094b-4ce5-a825-d5b3d3b18948 admin admin] PATH_INFO: `/v3/projects/1e0ee14bac364a66b46a76f74172c6ad/users/bd65b0c1f44c41db8f25fbc9848a9173/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:11.600415 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bfaab-094b-4ce5-a825-d5b3d3b18948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e0ee14bac364a66b46a76f74172c6ad, user_id=bd65b0c1f44c41db8f25fbc9848a9173, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:11.603881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bfaab-094b-4ce5-a825-d5b3d3b18948 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:11.617943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1366/2720] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:14:11 2022] PUT /identity/v3/projects/1e0ee14bac364a66b46a76f74172c6ad/users/bd65b0c1f44c41db8f25fbc9848a9173/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:14:12.638148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a91c2c-e3ca-4a30-b020-162fd1181ff0 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:12.640561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-44a91c2c-e3ca-4a30-b020-162fd1181ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:12.657964 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a91c2c-e3ca-4a30-b020-162fd1181ff0 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:12.730666 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d0f13-b9e4-44de-95ad-f3f840762238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:12.731553 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d0f13-b9e4-44de-95ad-f3f840762238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:12.731838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d0f13-b9e4-44de-95ad-f3f840762238 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:12.732113 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d0f13-b9e4-44de-95ad-f3f840762238 admin admin] PATH_INFO: `/v3/users/2fc5fd88556145fda1fb0067d5e9adcc` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:12.733330 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d0f13-b9e4-44de-95ad-f3f840762238 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc5fd88556145fda1fb0067d5e9adcc)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:12.735322 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d0f13-b9e4-44de-95ad-f3f840762238 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:12.958496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bfaab-094b-4ce5-a825-d5b3d3b18948 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:12.960353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-ff7bfaab-094b-4ce5-a825-d5b3d3b18948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:12.973887 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bfaab-094b-4ce5-a825-d5b3d3b18948 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:12.980370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941646d1-8b7e-44b7-a647-a94a065ff016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:12.985794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941646d1-8b7e-44b7-a647-a94a065ff016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:12.985794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941646d1-8b7e-44b7-a647-a94a065ff016 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:12.985794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941646d1-8b7e-44b7-a647-a94a065ff016 admin admin] PATH_INFO: `/v3/users/54f8741cfc7a4953b53470368207edfc` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:12.985794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941646d1-8b7e-44b7-a647-a94a065ff016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54f8741cfc7a4953b53470368207edfc)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:12.985794 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941646d1-8b7e-44b7-a647-a94a065ff016 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:12.992436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7e3d0f13-b9e4-44de-95ad-f3f840762238 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc5fd88556145fda1fb0067d5e9adcc'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:12.993591 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7e3d0f13-b9e4-44de-95ad-f3f840762238 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc5fd88556145fda1fb0067d5e9adcc'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:12.994407 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7e3d0f13-b9e4-44de-95ad-f3f840762238 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc5fd88556145fda1fb0067d5e9adcc'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:13.003292 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7e3d0f13-b9e4-44de-95ad-f3f840762238 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc5fd88556145fda1fb0067d5e9adcc'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:13.012307 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:13.018143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-7e3d0f13-b9e4-44de-95ad-f3f840762238 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc5fd88556145fda1fb0067d5e9adcc'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:13.118884 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1355/2721] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:14:12 2022] DELETE /identity/v3/users/2fc5fd88556145fda1fb0067d5e9adcc => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:14:13.230508 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:13.259124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-941646d1-8b7e-44b7-a647-a94a065ff016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54f8741cfc7a4953b53470368207edfc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:13.276701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-941646d1-8b7e-44b7-a647-a94a065ff016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54f8741cfc7a4953b53470368207edfc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:13.442811 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-941646d1-8b7e-44b7-a647-a94a065ff016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54f8741cfc7a4953b53470368207edfc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:13.468586 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-941646d1-8b7e-44b7-a647-a94a065ff016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54f8741cfc7a4953b53470368207edfc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:13.469314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-941646d1-8b7e-44b7-a647-a94a065ff016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54f8741cfc7a4953b53470368207edfc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:13.484618 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1367/2722] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:14:12 2022] DELETE /identity/v3/users/54f8741cfc7a4953b53470368207edfc => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:14:13.491054 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1489aa1-7007-45ae-91ad-8f1e27e9dedf None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:13.491586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1489aa1-7007-45ae-91ad-8f1e27e9dedf None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:13.491869 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1489aa1-7007-45ae-91ad-8f1e27e9dedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:13.513296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-f1489aa1-7007-45ae-91ad-8f1e27e9dedf None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:14:13.514562 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f1489aa1-7007-45ae-91ad-8f1e27e9dedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:13.557580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941646d1-8b7e-44b7-a647-a94a065ff016 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:13.560189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-941646d1-8b7e-44b7-a647-a94a065ff016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:13.660073 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1356/2723] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:14:13.665451 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1489aa1-7007-45ae-91ad-8f1e27e9dedf None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:13.672363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f1489aa1-7007-45ae-91ad-8f1e27e9dedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:13.676064 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941646d1-8b7e-44b7-a647-a94a065ff016 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:13.685349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:13.686011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:13.686290 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:13.686717 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] PATH_INFO: `/v3/projects/a9f0ab2fc6e64eada5f55a0f0a835073` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:13.688783 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9f0ab2fc6e64eada5f55a0f0a835073)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:13.691930 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:13.799193 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1489aa1-7007-45ae-91ad-8f1e27e9dedf None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:13.808281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e4045-4fcf-4b62-a987-b97e3c9e61c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:13.808929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e4045-4fcf-4b62-a987-b97e3c9e61c8 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:13.809410 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e4045-4fcf-4b62-a987-b97e3c9e61c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:13.809910 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e4045-4fcf-4b62-a987-b97e3c9e61c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:13.832596 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e4045-4fcf-4b62-a987-b97e3c9e61c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:13.834309 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e4045-4fcf-4b62-a987-b97e3c9e61c8 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:13.855358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9f0ab2fc6e64eada5f55a0f0a835073', 'request_id': 'req-4eb8b701-8e21-413c-903f-435c02d76c5b'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:13.856370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1357/2724] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:14:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:14:13.865083 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9f0ab2fc6e64eada5f55a0f0a835073', 'request_id': 'req-4eb8b701-8e21-413c-903f-435c02d76c5b'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:14.003189 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9f0ab2fc6e64eada5f55a0f0a835073', 'request_id': 'req-4eb8b701-8e21-413c-903f-435c02d76c5b'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:14.026066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] The token cache is being invalidate because project a9f0ab2fc6e64eada5f55a0f0a835073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:14:14.026390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:14.027103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:14.028900 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1368/2725] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:14:13 2022] DELETE /identity/v3/projects/a9f0ab2fc6e64eada5f55a0f0a835073 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:14:14.032821 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e4045-4fcf-4b62-a987-b97e3c9e61c8 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:14.048247 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e4045-4fcf-4b62-a987-b97e3c9e61c8 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:14.160913 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437406e7-59e4-480b-8a87-1703aafed223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:14.162205 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437406e7-59e4-480b-8a87-1703aafed223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:14.162499 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437406e7-59e4-480b-8a87-1703aafed223 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:14.163062 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437406e7-59e4-480b-8a87-1703aafed223 admin admin] PATH_INFO: `/v3/users/e49e9ffd54d747ab8d87dd7b7db9d741` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:14.164626 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437406e7-59e4-480b-8a87-1703aafed223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e49e9ffd54d747ab8d87dd7b7db9d741)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:14.166351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437406e7-59e4-480b-8a87-1703aafed223 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:14.363715 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:14.427543 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-437406e7-59e4-480b-8a87-1703aafed223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e49e9ffd54d747ab8d87dd7b7db9d741'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:14.429220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-437406e7-59e4-480b-8a87-1703aafed223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e49e9ffd54d747ab8d87dd7b7db9d741'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:14.432570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-437406e7-59e4-480b-8a87-1703aafed223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e49e9ffd54d747ab8d87dd7b7db9d741'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:14.444298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-437406e7-59e4-480b-8a87-1703aafed223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e49e9ffd54d747ab8d87dd7b7db9d741'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:14.483677 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-437406e7-59e4-480b-8a87-1703aafed223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e49e9ffd54d747ab8d87dd7b7db9d741'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:14.577921 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1358/2726] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:14:14 2022] DELETE /identity/v3/users/e49e9ffd54d747ab8d87dd7b7db9d741 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:14:14.613651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:14.615812 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:14.654162 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8b701-8e21-413c-903f-435c02d76c5b admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:14.779070 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:14.779810 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:14.780095 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:14.781057 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6 admin admin] PATH_INFO: `/v3/projects/76685d86ee9d408e9cc57e99f5896307` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:14.785840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76685d86ee9d408e9cc57e99f5896307)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:14.788555 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:15.980982 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437406e7-59e4-480b-8a87-1703aafed223 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:15.982463 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-437406e7-59e4-480b-8a87-1703aafed223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:16.007202 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437406e7-59e4-480b-8a87-1703aafed223 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:16.041580 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76685d86ee9d408e9cc57e99f5896307', 'request_id': 'req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:16.106552 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76685d86ee9d408e9cc57e99f5896307', 'request_id': 'req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:16.136606 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:16.137922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:16.137922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:16.137922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] PATH_INFO: `/v3/projects/88400cb6d31b453baa2ea82f51ecc6b9` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:16.139124 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88400cb6d31b453baa2ea82f51ecc6b9)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:16.141099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:16.237563 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76685d86ee9d408e9cc57e99f5896307', 'request_id': 'req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:16.272456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6 admin admin] The token cache is being invalidate because project 76685d86ee9d408e9cc57e99f5896307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:14:16.272456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:16.272456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c6d47b9c-591b-4cc6-ab18-48eb9bb625c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:16.272456 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1369/2727] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:14:14 2022] DELETE /identity/v3/projects/76685d86ee9d408e9cc57e99f5896307 => generated 0 bytes in 1657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:14:16.274736 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cc16a-176b-47de-ab42-1631ecb373be None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:16.275102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cc16a-176b-47de-ab42-1631ecb373be None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:16.275393 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cc16a-176b-47de-ab42-1631ecb373be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:16.328824 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-946cc16a-176b-47de-ab42-1631ecb373be None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:14:16.328824 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-946cc16a-176b-47de-ab42-1631ecb373be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:16.339902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88400cb6d31b453baa2ea82f51ecc6b9', 'request_id': 'req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:16.347397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88400cb6d31b453baa2ea82f51ecc6b9', 'request_id': 'req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:16.368628 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88400cb6d31b453baa2ea82f51ecc6b9', 'request_id': 'req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:16.403895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1370/2728] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:14:16.413831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bafa3f-5e3a-4ab1-b0ea-1f71ef8595fa None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:16.414141 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bafa3f-5e3a-4ab1-b0ea-1f71ef8595fa None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:16.414469 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bafa3f-5e3a-4ab1-b0ea-1f71ef8595fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:16.435695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-57bafa3f-5e3a-4ab1-b0ea-1f71ef8595fa None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:14:16.437033 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-57bafa3f-5e3a-4ab1-b0ea-1f71ef8595fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:16.452009 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1371/2729] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:14:16.453220 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bafa3f-5e3a-4ab1-b0ea-1f71ef8595fa None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:16.465175 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bafa3f-5e3a-4ab1-b0ea-1f71ef8595fa None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:16.472314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e4366-9294-4e4b-96f3-cecf0b38b337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:16.472314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e4366-9294-4e4b-96f3-cecf0b38b337 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:16.472314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e4366-9294-4e4b-96f3-cecf0b38b337 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:16.472314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e4366-9294-4e4b-96f3-cecf0b38b337 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:16.472729 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e4366-9294-4e4b-96f3-cecf0b38b337 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:14:16.472821 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e4366-9294-4e4b-96f3-cecf0b38b337 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:16.474361 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e4366-9294-4e4b-96f3-cecf0b38b337 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:16.487190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1372/2730] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:14:16.491601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e4366-9294-4e4b-96f3-cecf0b38b337 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:16.501913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e4366-9294-4e4b-96f3-cecf0b38b337 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:16.510142 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2c339-d310-4160-ac04-1ee3ca7a654f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:16.511378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2c339-d310-4160-ac04-1ee3ca7a654f admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:16.511378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2c339-d310-4160-ac04-1ee3ca7a654f admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:16.511378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2c339-d310-4160-ac04-1ee3ca7a654f admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:16.519910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2c339-d310-4160-ac04-1ee3ca7a654f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:16.519910 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2c339-d310-4160-ac04-1ee3ca7a654f admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:16.520183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] The token cache is being invalidate because project 88400cb6d31b453baa2ea82f51ecc6b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:14:16.520183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:16.520183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:16.520183 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1359/2731] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:14:15 2022] DELETE /identity/v3/projects/88400cb6d31b453baa2ea82f51ecc6b9 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:14:16.687540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1373/2732] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:14:16 2022] POST /identity/v3/projects => generated 399 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:14:16.692482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:16.694753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:16.715279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1037d9-f3a8-49f8-911a-99ad864c8ea5 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:16.783043 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e285128-bd94-43ff-9682-520b98fefea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:16.783863 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e285128-bd94-43ff-9682-520b98fefea2 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:16.787064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e285128-bd94-43ff-9682-520b98fefea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:16.787064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e285128-bd94-43ff-9682-520b98fefea2 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:16.787064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e285128-bd94-43ff-9682-520b98fefea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:16.787520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e285128-bd94-43ff-9682-520b98fefea2 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:16.943266 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1360/2733] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:14:16 2022] POST /identity/v3/users => generated 300 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:14:16.947354 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2c339-d310-4160-ac04-1ee3ca7a654f admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:16.958715 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2c339-d310-4160-ac04-1ee3ca7a654f admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:16.964017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a397f9-5c9e-4f55-a8ef-b0f40f677056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:16.964647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a397f9-5c9e-4f55-a8ef-b0f40f677056 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:16.964953 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a397f9-5c9e-4f55-a8ef-b0f40f677056 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:16.965232 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a397f9-5c9e-4f55-a8ef-b0f40f677056 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:16.966048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a397f9-5c9e-4f55-a8ef-b0f40f677056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:16.967695 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a397f9-5c9e-4f55-a8ef-b0f40f677056 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:16.978800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1374/2734] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:14:16.982927 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e285128-bd94-43ff-9682-520b98fefea2 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:16.995070 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e285128-bd94-43ff-9682-520b98fefea2 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:16.999865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380ab57-7e92-40e7-bc2a-fb81ee14f4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:17.000569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380ab57-7e92-40e7-bc2a-fb81ee14f4ed admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:17.000849 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380ab57-7e92-40e7-bc2a-fb81ee14f4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:17.001125 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380ab57-7e92-40e7-bc2a-fb81ee14f4ed admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:17.001900 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380ab57-7e92-40e7-bc2a-fb81ee14f4ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:17.003573 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380ab57-7e92-40e7-bc2a-fb81ee14f4ed admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:17.015964 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1361/2735] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:14:17.020492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a397f9-5c9e-4f55-a8ef-b0f40f677056 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:17.031513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a397f9-5c9e-4f55-a8ef-b0f40f677056 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:17.036374 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4a7d6-518b-47af-8dee-93ecf1e8ed00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:17.037058 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4a7d6-518b-47af-8dee-93ecf1e8ed00 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:17.037350 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4a7d6-518b-47af-8dee-93ecf1e8ed00 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:17.037632 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4a7d6-518b-47af-8dee-93ecf1e8ed00 admin admin] PATH_INFO: `/v3/projects/a99fa9f875a943b38560abdfb123f328/users/33a50ece4cc04d90a6520a14c4b51ac0/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:17.054712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4a7d6-518b-47af-8dee-93ecf1e8ed00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a99fa9f875a943b38560abdfb123f328, user_id=33a50ece4cc04d90a6520a14c4b51ac0, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:17.058372 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4a7d6-518b-47af-8dee-93ecf1e8ed00 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:17.163415 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1375/2736] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:14:17 2022] PUT /identity/v3/projects/a99fa9f875a943b38560abdfb123f328/users/33a50ece4cc04d90a6520a14c4b51ac0/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:14:17.176732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872920fe-d7f9-46d4-91d5-bd57ff889f52 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:17.178648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872920fe-d7f9-46d4-91d5-bd57ff889f52 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:17.179145 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872920fe-d7f9-46d4-91d5-bd57ff889f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:17.232951 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-872920fe-d7f9-46d4-91d5-bd57ff889f52 None None] MFA Rules not processed for user `33a50ece4cc04d90a6520a14c4b51ac0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:14:17.234814 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-872920fe-d7f9-46d4-91d5-bd57ff889f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:17.336762 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1362/2737] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:14:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:14:17.349112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4a7d6-518b-47af-8dee-93ecf1e8ed00 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:17.353472 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a8a4a7d6-518b-47af-8dee-93ecf1e8ed00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:17.423613 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4a7d6-518b-47af-8dee-93ecf1e8ed00 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:17.428721 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1692e-8a8f-4b90-a802-701e236057ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:17.433965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1692e-8a8f-4b90-a802-701e236057ae service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:17.433965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1692e-8a8f-4b90-a802-701e236057ae service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:17.433965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1692e-8a8f-4b90-a802-701e236057ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:17.449007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1692e-8a8f-4b90-a802-701e236057ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:17.450753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1692e-8a8f-4b90-a802-701e236057ae service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:17.463348 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1376/2738] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:14:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:14:17.643596 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:19.072396 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:19.455797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb97c8-4017-4ac0-b671-ca0f3cb95183 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:19.456091 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb97c8-4017-4ac0-b671-ca0f3cb95183 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:19.456549 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb97c8-4017-4ac0-b671-ca0f3cb95183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:19.486630 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-fefb97c8-4017-4ac0-b671-ca0f3cb95183 None None] MFA Rules not processed for user `bd65b0c1f44c41db8f25fbc9848a9173`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:14:19.488571 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-fefb97c8-4017-4ac0-b671-ca0f3cb95183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:19.573343 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1363/2739] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:14:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:14:19.588264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1692e-8a8f-4b90-a802-701e236057ae service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:19.591905 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-cca1692e-8a8f-4b90-a802-701e236057ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:19.732270 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1692e-8a8f-4b90-a802-701e236057ae service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:19.740956 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb42a6-8426-429e-a7bb-0862d6ddd105 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:19.741907 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb42a6-8426-429e-a7bb-0862d6ddd105 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:19.743635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb42a6-8426-429e-a7bb-0862d6ddd105 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:19.743919 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb42a6-8426-429e-a7bb-0862d6ddd105 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:19.745925 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b7cb42a6-8426-429e-a7bb-0862d6ddd105 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:19.851252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb42a6-8426-429e-a7bb-0862d6ddd105 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:19.853196 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb42a6-8426-429e-a7bb-0862d6ddd105 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:19.874163 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1377/2740] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:14:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:14:20.462056 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:24.738823 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c33f65-ffa3-46cb-8e7d-d335d9acc63c None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:24.739327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c33f65-ffa3-46cb-8e7d-d335d9acc63c None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:24.739559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c33f65-ffa3-46cb-8e7d-d335d9acc63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:24.760378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-25c33f65-ffa3-46cb-8e7d-d335d9acc63c None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:14:24.763445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-25c33f65-ffa3-46cb-8e7d-d335d9acc63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:24.790518 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:24.824432 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1364/2741] 10.222.0.94 () {58 vars in 967 bytes} [Tue Jul 26 15:14:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:14:24.837565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb42a6-8426-429e-a7bb-0862d6ddd105 service nova] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:24.839078 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b7cb42a6-8426-429e-a7bb-0862d6ddd105 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:24.891482 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=100 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:24.921983 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb42a6-8426-429e-a7bb-0862d6ddd105 service nova] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:24.939726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dea19e-5c5e-406f-87b8-57efa0a1c8f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:24.939726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dea19e-5c5e-406f-87b8-57efa0a1c8f7 service swift] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:24.939726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dea19e-5c5e-406f-87b8-57efa0a1c8f7 service swift] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:24.939726 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dea19e-5c5e-406f-87b8-57efa0a1c8f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:24.940628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dea19e-5c5e-406f-87b8-57efa0a1c8f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:24.940628 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dea19e-5c5e-406f-87b8-57efa0a1c8f7 service swift] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:24.949344 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1378/2742] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:14:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:14:27.069193 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c33f65-ffa3-46cb-8e7d-d335d9acc63c None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:27.140111 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c33f65-ffa3-46cb-8e7d-d335d9acc63c None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:27.144690 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4043c05-141b-42f4-9159-2715d4e4eed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:27.148470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4043c05-141b-42f4-9159-2715d4e4eed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:27.148470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4043c05-141b-42f4-9159-2715d4e4eed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:27.148470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4043c05-141b-42f4-9159-2715d4e4eed8 admin admin] PATH_INFO: `/v3/users/33a50ece4cc04d90a6520a14c4b51ac0` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:27.148470 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4043c05-141b-42f4-9159-2715d4e4eed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a50ece4cc04d90a6520a14c4b51ac0)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:27.149827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4043c05-141b-42f4-9159-2715d4e4eed8 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:27.452726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b4043c05-141b-42f4-9159-2715d4e4eed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a50ece4cc04d90a6520a14c4b51ac0'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:27.453406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b4043c05-141b-42f4-9159-2715d4e4eed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a50ece4cc04d90a6520a14c4b51ac0'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:27.454008 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b4043c05-141b-42f4-9159-2715d4e4eed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a50ece4cc04d90a6520a14c4b51ac0'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:27.462108 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b4043c05-141b-42f4-9159-2715d4e4eed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a50ece4cc04d90a6520a14c4b51ac0'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:27.474256 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b4043c05-141b-42f4-9159-2715d4e4eed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a50ece4cc04d90a6520a14c4b51ac0'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:27.579671 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1365/2743] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:14:27 2022] DELETE /identity/v3/users/33a50ece4cc04d90a6520a14c4b51ac0 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:14:27.584170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006305e-79e6-4438-80e7-6ae828053b96 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:27.584710 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006305e-79e6-4438-80e7-6ae828053b96 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:27.584710 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006305e-79e6-4438-80e7-6ae828053b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:27.605105 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-a006305e-79e6-4438-80e7-6ae828053b96 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:14:27.606321 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a006305e-79e6-4438-80e7-6ae828053b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:27.671198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1379/2744] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:14:27.683895 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4043c05-141b-42f4-9159-2715d4e4eed8 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:27.685363 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b4043c05-141b-42f4-9159-2715d4e4eed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:27.750063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4043c05-141b-42f4-9159-2715d4e4eed8 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:27.753803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e81cfb-8e63-4159-b733-25a2b405f4dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:27.754440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e81cfb-8e63-4159-b733-25a2b405f4dd service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:27.754726 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e81cfb-8e63-4159-b733-25a2b405f4dd service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:27.755006 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e81cfb-8e63-4159-b733-25a2b405f4dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:27.763572 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:27.766282 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e81cfb-8e63-4159-b733-25a2b405f4dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:27.767850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e81cfb-8e63-4159-b733-25a2b405f4dd service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:27.780203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1366/2745] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:14:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:14:28.233434 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006305e-79e6-4438-80e7-6ae828053b96 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:28.235346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a006305e-79e6-4438-80e7-6ae828053b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:28.251779 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006305e-79e6-4438-80e7-6ae828053b96 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:28.256706 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:28.257356 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:28.258216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:28.258541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] PATH_INFO: `/v3/projects/a99fa9f875a943b38560abdfb123f328` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:28.259829 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a99fa9f875a943b38560abdfb123f328)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:28.261750 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:28.407389 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a99fa9f875a943b38560abdfb123f328', 'request_id': 'req-d4755b92-5eae-4715-bed1-fc56479eb9a4'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:28.412998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a99fa9f875a943b38560abdfb123f328', 'request_id': 'req-d4755b92-5eae-4715-bed1-fc56479eb9a4'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:28.488214 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a99fa9f875a943b38560abdfb123f328', 'request_id': 'req-d4755b92-5eae-4715-bed1-fc56479eb9a4'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:28.502053 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] The token cache is being invalidate because project a99fa9f875a943b38560abdfb123f328 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:14:28.502325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:28.503230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:14:28.504976 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1380/2746] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:14:28 2022] DELETE /identity/v3/projects/a99fa9f875a943b38560abdfb123f328 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:14:37.933619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:38.498707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:41.377421 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb65bc3-a0ca-491b-bde5-804b2938b5f8 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:41.378010 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb65bc3-a0ca-491b-bde5-804b2938b5f8 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:41.378010 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb65bc3-a0ca-491b-bde5-804b2938b5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:41.398022 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-9cb65bc3-a0ca-491b-bde5-804b2938b5f8 None None] MFA Rules not processed for user `c11f43d5614d46cd996789819075520d`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:14:41.400084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9cb65bc3-a0ca-491b-bde5-804b2938b5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:41.456902 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1367/2747] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:14:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:14:45.899918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:45.901433 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:45.961694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4755b92-5eae-4715-bed1-fc56479eb9a4 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:45.966179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a18ff1-c8f6-49ff-a61d-aa0dc185aa1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:45.966806 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a18ff1-c8f6-49ff-a61d-aa0dc185aa1f service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:45.967252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a18ff1-c8f6-49ff-a61d-aa0dc185aa1f service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:45.967535 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a18ff1-c8f6-49ff-a61d-aa0dc185aa1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:45.969401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-73a18ff1-c8f6-49ff-a61d-aa0dc185aa1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:46.043219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a18ff1-c8f6-49ff-a61d-aa0dc185aa1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:46.044740 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a18ff1-c8f6-49ff-a61d-aa0dc185aa1f service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:46.056656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1381/2748] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:14:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:14:47.167740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb65bc3-a0ca-491b-bde5-804b2938b5f8 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:14:47.169158 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9cb65bc3-a0ca-491b-bde5-804b2938b5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:47.182140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb65bc3-a0ca-491b-bde5-804b2938b5f8 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:14:47.186925 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54628b-509c-4a1a-ac7c-1ba99fbbc3da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:14:47.186925 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54628b-509c-4a1a-ac7c-1ba99fbbc3da service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:14:47.187268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54628b-509c-4a1a-ac7c-1ba99fbbc3da service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:14:47.187268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54628b-509c-4a1a-ac7c-1ba99fbbc3da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:14:47.189109 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1a54628b-509c-4a1a-ac7c-1ba99fbbc3da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:14:47.203872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a54628b-509c-4a1a-ac7c-1ba99fbbc3da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:14:47.205240 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a54628b-509c-4a1a-ac7c-1ba99fbbc3da service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:14:47.216783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1368/2749] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:14:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:14:56.051016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=54 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:14:57.211794 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=31 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:02.921981 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a18ff1-c8f6-49ff-a61d-aa0dc185aa1f service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:02.930881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a18ff1-c8f6-49ff-a61d-aa0dc185aa1f service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:02.935282 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed1f6c-2ebb-463b-b945-16e01844ab29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:02.935967 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed1f6c-2ebb-463b-b945-16e01844ab29 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:02.936295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed1f6c-2ebb-463b-b945-16e01844ab29 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:02.936513 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed1f6c-2ebb-463b-b945-16e01844ab29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:02.938293 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-09ed1f6c-2ebb-463b-b945-16e01844ab29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:03.002746 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed1f6c-2ebb-463b-b945-16e01844ab29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:03.004207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed1f6c-2ebb-463b-b945-16e01844ab29 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:03.014480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1382/2750] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:15:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:15:13.009450 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=30 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:17.822514 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439b2bd-132c-4530-9a8a-34a302f2d6b5 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:17.823095 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439b2bd-132c-4530-9a8a-34a302f2d6b5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:17.823095 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439b2bd-132c-4530-9a8a-34a302f2d6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:17.842547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-d439b2bd-132c-4530-9a8a-34a302f2d6b5 None None] MFA Rules not processed for user `b7e467910de54b7ea55cb2d728a8c9be`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:15:17.843778 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d439b2bd-132c-4530-9a8a-34a302f2d6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:17.905812 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1369/2751] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:15:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:15:27.896687 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=25 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:28.310280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed1f6c-2ebb-463b-b945-16e01844ab29 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:28.320525 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed1f6c-2ebb-463b-b945-16e01844ab29 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:28.325496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a6285-505b-46ff-8000-e3095b53b6aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:28.325496 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6285-505b-46ff-8000-e3095b53b6aa service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:28.325829 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6285-505b-46ff-8000-e3095b53b6aa service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:28.325829 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6285-505b-46ff-8000-e3095b53b6aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:28.327659 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e60a6285-505b-46ff-8000-e3095b53b6aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:28.339048 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a6285-505b-46ff-8000-e3095b53b6aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:28.341238 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a6285-505b-46ff-8000-e3095b53b6aa service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:28.349949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1383/2752] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:15:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:15:32.009590 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439b2bd-132c-4530-9a8a-34a302f2d6b5 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:15:32.011045 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d439b2bd-132c-4530-9a8a-34a302f2d6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:32.027207 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439b2bd-132c-4530-9a8a-34a302f2d6b5 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:15:32.033160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8274eb2-6b7e-4900-86bf-434c3d8626c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:15:32.037457 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8274eb2-6b7e-4900-86bf-434c3d8626c1 service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:15:32.037457 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8274eb2-6b7e-4900-86bf-434c3d8626c1 service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:15:32.037457 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8274eb2-6b7e-4900-86bf-434c3d8626c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:15:32.037457 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f8274eb2-6b7e-4900-86bf-434c3d8626c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:15:32.049560 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8274eb2-6b7e-4900-86bf-434c3d8626c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:15:32.051212 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8274eb2-6b7e-4900-86bf-434c3d8626c1 service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:15:32.063079 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1370/2753] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:15:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:15:38.346841 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=6 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:15:42.057628 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=6 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:24.744374 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7458b5-edae-45ec-b0a2-5a26c72d4a01 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:24.744971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7458b5-edae-45ec-b0a2-5a26c72d4a01 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:24.744971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7458b5-edae-45ec-b0a2-5a26c72d4a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:24.808716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-aa7458b5-edae-45ec-b0a2-5a26c72d4a01 None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:24.810041 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-aa7458b5-edae-45ec-b0a2-5a26c72d4a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:24.881976 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1384/2754] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 15:16:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:16:25.051531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8274eb2-6b7e-4900-86bf-434c3d8626c1 service nova] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:25.053056 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f8274eb2-6b7e-4900-86bf-434c3d8626c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:25.165465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8274eb2-6b7e-4900-86bf-434c3d8626c1 service nova] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:25.172752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d7e2d-698f-4651-b18a-1da44e1c4f1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:25.173388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d7e2d-698f-4651-b18a-1da44e1c4f1a service swift] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:25.173706 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d7e2d-698f-4651-b18a-1da44e1c4f1a service swift] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:25.174014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d7e2d-698f-4651-b18a-1da44e1c4f1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:25.183211 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d7e2d-698f-4651-b18a-1da44e1c4f1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:25.184832 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d7e2d-698f-4651-b18a-1da44e1c4f1a service swift] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:25.194377 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1371/2755] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:16:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:16:26.171685 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7458b5-edae-45ec-b0a2-5a26c72d4a01 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:26.173782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-aa7458b5-edae-45ec-b0a2-5a26c72d4a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:26.191446 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7458b5-edae-45ec-b0a2-5a26c72d4a01 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:26.197119 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121f9c8-d271-4141-99b6-c36cd310eb8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:26.197756 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121f9c8-d271-4141-99b6-c36cd310eb8b service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:26.198039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121f9c8-d271-4141-99b6-c36cd310eb8b service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:26.198312 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121f9c8-d271-4141-99b6-c36cd310eb8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:26.200159 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-a121f9c8-d271-4141-99b6-c36cd310eb8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:26.217042 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121f9c8-d271-4141-99b6-c36cd310eb8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:26.220259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121f9c8-d271-4141-99b6-c36cd310eb8b service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:26.241249 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1385/2756] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:16:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:16:35.190384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=41 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:36.232646 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=46 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:42.983747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab51e47-a01e-4037-ba89-b87c4bbf111c None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:42.984249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab51e47-a01e-4037-ba89-b87c4bbf111c None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:42.984358 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab51e47-a01e-4037-ba89-b87c4bbf111c None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:42.991035 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1372/2757] 10.222.0.94 () {54 vars in 921 bytes} [Tue Jul 26 15:16:42 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:16:43.000293 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bbe6a-534e-48da-bf07-7c58a62aa2ee None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.000857 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bbe6a-534e-48da-bf07-7c58a62aa2ee None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.000995 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bbe6a-534e-48da-bf07-7c58a62aa2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.045749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-d13bbe6a-534e-48da-bf07-7c58a62aa2ee None None] MFA Rules not processed for user `6a4278fe995e432d88f4346d6f54218b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:43.047001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d13bbe6a-534e-48da-bf07-7c58a62aa2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:43.069206 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1386/2758] 10.222.0.94 () {58 vars in 1018 bytes} [Tue Jul 26 15:16:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:16:43.082292 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab51e47-a01e-4037-ba89-b87c4bbf111c None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:43.084017 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9ab51e47-a01e-4037-ba89-b87c4bbf111c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:43.172751 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab51e47-a01e-4037-ba89-b87c4bbf111c None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:43.176388 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc6247-befb-42d3-a447-0aaba1ed4389 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b153935910c142aa80792e111872318a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:43.176998 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc6247-befb-42d3-a447-0aaba1ed4389 service glance] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.177281 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc6247-befb-42d3-a447-0aaba1ed4389 service glance] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.177562 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc6247-befb-42d3-a447-0aaba1ed4389 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.189392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc6247-befb-42d3-a447-0aaba1ed4389 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:43.190833 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc6247-befb-42d3-a447-0aaba1ed4389 service glance] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:43.203798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1373/2759] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:16:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:16:43.442218 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0afaa8-22a2-4370-bd22-f29163fcdfda None None] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.442437 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0afaa8-22a2-4370-bd22-f29163fcdfda None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.442704 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0afaa8-22a2-4370-bd22-f29163fcdfda None None] PATH_INFO: `/` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.445129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1387/2760] 10.222.0.94 () {54 vars in 921 bytes} [Tue Jul 26 15:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:16:43.451328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015c499-eb5a-41b5-981f-b343785b752a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.451328 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015c499-eb5a-41b5-981f-b343785b752a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.451684 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015c499-eb5a-41b5-981f-b343785b752a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.493521 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-d015c499-eb5a-41b5-981f-b343785b752a None None] MFA Rules not processed for user `d39fd2708591424ca4892973e7da48ee`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:43.494739 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d015c499-eb5a-41b5-981f-b343785b752a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:43.501339 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1374/2761] 10.222.0.94 () {58 vars in 1018 bytes} [Tue Jul 26 15:16:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:16:43.514029 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0afaa8-22a2-4370-bd22-f29163fcdfda None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:43.526078 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0afaa8-22a2-4370-bd22-f29163fcdfda None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:43.530204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab39d6-8af9-43aa-84a2-363e046d6ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:43.530829 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab39d6-8af9-43aa-84a2-363e046d6ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.531111 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab39d6-8af9-43aa-84a2-363e046d6ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.531411 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab39d6-8af9-43aa-84a2-363e046d6ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.539428 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab39d6-8af9-43aa-84a2-363e046d6ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:43.540942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab39d6-8af9-43aa-84a2-363e046d6ef5 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:43.549709 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1388/2762] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:16:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:16:43.639103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb46b69-e9da-440d-bd9b-2774a388812a None None] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.648565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb46b69-e9da-440d-bd9b-2774a388812a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.648565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb46b69-e9da-440d-bd9b-2774a388812a None None] PATH_INFO: `/` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.648565 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1375/2763] 10.222.0.94 () {54 vars in 921 bytes} [Tue Jul 26 15:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 15:16:43.649191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b776a16-8073-48d9-b8c3-536036aa091b None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.649820 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b776a16-8073-48d9-b8c3-536036aa091b None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.650217 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b776a16-8073-48d9-b8c3-536036aa091b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.676108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-1b776a16-8073-48d9-b8c3-536036aa091b None None] MFA Rules not processed for user `6a4278fe995e432d88f4346d6f54218b`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:43.677724 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1b776a16-8073-48d9-b8c3-536036aa091b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:43.692531 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1389/2764] 10.222.0.94 () {58 vars in 1018 bytes} [Tue Jul 26 15:16:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:16:43.703394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb46b69-e9da-440d-bd9b-2774a388812a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:43.715536 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb46b69-e9da-440d-bd9b-2774a388812a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:43.719646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a686c97-cc29-4eef-b35d-0f6e78a8841d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:43.720301 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a686c97-cc29-4eef-b35d-0f6e78a8841d service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.720619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a686c97-cc29-4eef-b35d-0f6e78a8841d service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.721106 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a686c97-cc29-4eef-b35d-0f6e78a8841d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.733661 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a686c97-cc29-4eef-b35d-0f6e78a8841d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:43.735063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a686c97-cc29-4eef-b35d-0f6e78a8841d service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:43.751850 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1376/2765] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:16:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:16:43.826404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b776a16-8073-48d9-b8c3-536036aa091b None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:43.828845 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1b776a16-8073-48d9-b8c3-536036aa091b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:43.939824 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b776a16-8073-48d9-b8c3-536036aa091b None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:43.951073 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0221657-d53f-488b-8e6b-192a73c2d940 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a0728e2f6249bdb45d6f44fb15aa4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:43.952272 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0221657-d53f-488b-8e6b-192a73c2d940 service placement] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:43.953001 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0221657-d53f-488b-8e6b-192a73c2d940 service placement] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:43.953741 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0221657-d53f-488b-8e6b-192a73c2d940 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:43.958793 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e0221657-d53f-488b-8e6b-192a73c2d940 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:43.977047 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0221657-d53f-488b-8e6b-192a73c2d940 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:43.978677 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0221657-d53f-488b-8e6b-192a73c2d940 service placement] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:43.991555 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1390/2766] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:16:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:16:46.411557 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7f8e7-b572-43d6-847c-b8b64988f98d None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:46.412127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7f8e7-b572-43d6-847c-b8b64988f98d None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:46.412229 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7f8e7-b572-43d6-847c-b8b64988f98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:46.433733 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-2ca7f8e7-b572-43d6-847c-b8b64988f98d None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:16:46.434953 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-2ca7f8e7-b572-43d6-847c-b8b64988f98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:16:46.442380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1377/2767] 10.222.0.94 () {58 vars in 1009 bytes} [Tue Jul 26 15:16:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:16:46.462788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0221657-d53f-488b-8e6b-192a73c2d940 service placement] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:16:46.467530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0221657-d53f-488b-8e6b-192a73c2d940 service placement] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:16:46.473441 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11156fe7-5cec-431b-b106-3fb4dc6c814c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:16:46.474096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11156fe7-5cec-431b-b106-3fb4dc6c814c service swift] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:16:46.474380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11156fe7-5cec-431b-b106-3fb4dc6c814c service swift] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:16:46.474655 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11156fe7-5cec-431b-b106-3fb4dc6c814c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:16:46.486473 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11156fe7-5cec-431b-b106-3fb4dc6c814c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:16:46.488105 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11156fe7-5cec-431b-b106-3fb4dc6c814c service swift] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:16:46.498162 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1391/2768] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:16:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:16:56.427721 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=53 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:16:56.494928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=56 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:02.344253 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcf8b0-60f1-42d0-ba3c-832e39e93481 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:02.344789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcf8b0-60f1-42d0-ba3c-832e39e93481 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:02.344789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcf8b0-60f1-42d0-ba3c-832e39e93481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:02.366765 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-9bfcf8b0-60f1-42d0-ba3c-832e39e93481 None None] MFA Rules not processed for user `5b24d190536a4953930432287dfbfb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:02.368040 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-9bfcf8b0-60f1-42d0-ba3c-832e39e93481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:02.377039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1378/2769] 10.222.0.94 () {58 vars in 967 bytes} [Tue Jul 26 15:17:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:17:02.389700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11156fe7-5cec-431b-b106-3fb4dc6c814c service swift] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:02.401259 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11156fe7-5cec-431b-b106-3fb4dc6c814c service swift] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:02.406788 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505821e-5782-4daa-a1f4-6f92594d9f7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cea7b5ba4a4acabe8a26837cd97310', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:02.407405 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505821e-5782-4daa-a1f4-6f92594d9f7e service swift] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:02.407713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505821e-5782-4daa-a1f4-6f92594d9f7e service swift] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:02.407987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505821e-5782-4daa-a1f4-6f92594d9f7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:02.417625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505821e-5782-4daa-a1f4-6f92594d9f7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:02.419198 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505821e-5782-4daa-a1f4-6f92594d9f7e service swift] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:02.427944 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1392/2770] 10.222.0.94 () {60 vars in 1384 bytes} [Tue Jul 26 15:17:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 15:17:09.963356 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245cecf-d2bf-44f4-8fcd-41e091b1e4d5 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:09.963929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245cecf-d2bf-44f4-8fcd-41e091b1e4d5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:09.963929 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245cecf-d2bf-44f4-8fcd-41e091b1e4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:09.985441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-4245cecf-d2bf-44f4-8fcd-41e091b1e4d5 None None] MFA Rules not processed for user `c11f43d5614d46cd996789819075520d`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:09.986705 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4245cecf-d2bf-44f4-8fcd-41e091b1e4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:09.994084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1379/2771] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:17:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:17:10.320753 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505821e-5782-4daa-a1f4-6f92594d9f7e service swift] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:10.330480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505821e-5782-4daa-a1f4-6f92594d9f7e service swift] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:10.334237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a9c6a-da02-4179-801f-57bc0913528b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:10.334932 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9c6a-da02-4179-801f-57bc0913528b service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:10.335209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9c6a-da02-4179-801f-57bc0913528b service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:10.335506 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9c6a-da02-4179-801f-57bc0913528b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:10.337269 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-fd4a9c6a-da02-4179-801f-57bc0913528b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:10.347806 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a9c6a-da02-4179-801f-57bc0913528b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:10.350945 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a9c6a-da02-4179-801f-57bc0913528b service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:10.357752 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1393/2772] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:17:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:17:19.979217 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=6 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:20.353781 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=12 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:37.679075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245cecf-d2bf-44f4-8fcd-41e091b1e4d5 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:37.680552 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4245cecf-d2bf-44f4-8fcd-41e091b1e4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:37.740019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245cecf-d2bf-44f4-8fcd-41e091b1e4d5 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:37.743793 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d176fe-790a-43f5-a77e-ad23f4fdc2e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:37.744372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d176fe-790a-43f5-a77e-ad23f4fdc2e1 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:37.744658 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d176fe-790a-43f5-a77e-ad23f4fdc2e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:37.744946 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d176fe-790a-43f5-a77e-ad23f4fdc2e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:37.747586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-41d176fe-790a-43f5-a77e-ad23f4fdc2e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:37.759039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d176fe-790a-43f5-a77e-ad23f4fdc2e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:37.760503 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d176fe-790a-43f5-a77e-ad23f4fdc2e1 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:37.770144 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1380/2773] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:17:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:17:46.702441 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a9c6a-da02-4179-801f-57bc0913528b service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:46.712032 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a9c6a-da02-4179-801f-57bc0913528b service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:46.716804 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d5452-0dcb-4783-b46b-a896d7daaab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:46.717425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d5452-0dcb-4783-b46b-a896d7daaab6 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:46.717705 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d5452-0dcb-4783-b46b-a896d7daaab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:46.717979 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d5452-0dcb-4783-b46b-a896d7daaab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:46.720213 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-595d5452-0dcb-4783-b46b-a896d7daaab6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:46.791831 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d5452-0dcb-4783-b46b-a896d7daaab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:46.793637 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d5452-0dcb-4783-b46b-a896d7daaab6 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:46.804479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1394/2774] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:17:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:17:47.765718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=26 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:50.622196 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d176fe-790a-43f5-a77e-ad23f4fdc2e1 service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:50.624594 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-41d176fe-790a-43f5-a77e-ad23f4fdc2e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:50.645963 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d176fe-790a-43f5-a77e-ad23f4fdc2e1 service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:50.651060 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49db189-3878-4209-883f-a3ddc47726fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:50.652586 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49db189-3878-4209-883f-a3ddc47726fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:50.653029 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49db189-3878-4209-883f-a3ddc47726fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:50.653370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49db189-3878-4209-883f-a3ddc47726fa admin admin] PATH_INFO: `/v3/users/b7e467910de54b7ea55cb2d728a8c9be` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:50.654564 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49db189-3878-4209-883f-a3ddc47726fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e467910de54b7ea55cb2d728a8c9be)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:50.656577 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49db189-3878-4209-883f-a3ddc47726fa admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:50.906796 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e49db189-3878-4209-883f-a3ddc47726fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e467910de54b7ea55cb2d728a8c9be'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:50.907772 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e49db189-3878-4209-883f-a3ddc47726fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e467910de54b7ea55cb2d728a8c9be'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:50.908619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e49db189-3878-4209-883f-a3ddc47726fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e467910de54b7ea55cb2d728a8c9be'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:50.919423 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e49db189-3878-4209-883f-a3ddc47726fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e467910de54b7ea55cb2d728a8c9be'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:50.936041 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e49db189-3878-4209-883f-a3ddc47726fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e467910de54b7ea55cb2d728a8c9be'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:51.047208 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1381/2775] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:17:50 2022] DELETE /identity/v3/users/b7e467910de54b7ea55cb2d728a8c9be => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:17:51.424106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d5452-0dcb-4783-b46b-a896d7daaab6 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:51.443149 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d5452-0dcb-4783-b46b-a896d7daaab6 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:51.521419 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e73940-4cda-497c-a443-282d20ae94e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:51.522110 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e73940-4cda-497c-a443-282d20ae94e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:51.522352 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e73940-4cda-497c-a443-282d20ae94e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:51.522627 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e73940-4cda-497c-a443-282d20ae94e6 admin admin] PATH_INFO: `/v3/projects/b57441fed5b5437581930346bf9116ef` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:51.523843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e73940-4cda-497c-a443-282d20ae94e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b57441fed5b5437581930346bf9116ef)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:51.525735 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e73940-4cda-497c-a443-282d20ae94e6 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:51.645046 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-87e73940-4cda-497c-a443-282d20ae94e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b57441fed5b5437581930346bf9116ef', 'request_id': 'req-87e73940-4cda-497c-a443-282d20ae94e6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:51.652075 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-87e73940-4cda-497c-a443-282d20ae94e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b57441fed5b5437581930346bf9116ef', 'request_id': 'req-87e73940-4cda-497c-a443-282d20ae94e6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:51.736380 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-87e73940-4cda-497c-a443-282d20ae94e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b57441fed5b5437581930346bf9116ef', 'request_id': 'req-87e73940-4cda-497c-a443-282d20ae94e6'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:51.750049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-87e73940-4cda-497c-a443-282d20ae94e6 admin admin] The token cache is being invalidate because project b57441fed5b5437581930346bf9116ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:17:51.750336 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-87e73940-4cda-497c-a443-282d20ae94e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:51.751036 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-87e73940-4cda-497c-a443-282d20ae94e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:51.752643 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1395/2776] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:17:51 2022] DELETE /identity/v3/projects/b57441fed5b5437581930346bf9116ef => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:17:51.757526 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066693d2-350e-42e9-ac3c-bbed5f87c47d None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:51.758110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066693d2-350e-42e9-ac3c-bbed5f87c47d None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:51.758110 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066693d2-350e-42e9-ac3c-bbed5f87c47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:51.778535 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-066693d2-350e-42e9-ac3c-bbed5f87c47d None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:51.779961 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-066693d2-350e-42e9-ac3c-bbed5f87c47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:51.848260 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1382/2777] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:17:51.857647 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79526d99-9e11-4b7e-9955-bb2c30e1d15f None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:51.857936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79526d99-9e11-4b7e-9955-bb2c30e1d15f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:51.858215 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79526d99-9e11-4b7e-9955-bb2c30e1d15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:51.879042 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-79526d99-9e11-4b7e-9955-bb2c30e1d15f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:51.880289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-79526d99-9e11-4b7e-9955-bb2c30e1d15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:51.889006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1396/2778] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:17:51.893370 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066693d2-350e-42e9-ac3c-bbed5f87c47d None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:51.904246 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066693d2-350e-42e9-ac3c-bbed5f87c47d None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:51.908971 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba8995-edd2-4c53-ac02-80262078af41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:51.909583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba8995-edd2-4c53-ac02-80262078af41 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:51.909866 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba8995-edd2-4c53-ac02-80262078af41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:51.910143 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba8995-edd2-4c53-ac02-80262078af41 admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:51.910846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba8995-edd2-4c53-ac02-80262078af41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:51.911155 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba8995-edd2-4c53-ac02-80262078af41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:51.912783 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba8995-edd2-4c53-ac02-80262078af41 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:51.926986 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1383/2779] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 15:17:51.931060 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79526d99-9e11-4b7e-9955-bb2c30e1d15f None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:51.940610 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79526d99-9e11-4b7e-9955-bb2c30e1d15f None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:51.945534 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41457684-37a1-4455-b378-3093950686c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:51.946146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41457684-37a1-4455-b378-3093950686c5 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:51.946420 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41457684-37a1-4455-b378-3093950686c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:51.946689 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41457684-37a1-4455-b378-3093950686c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:51.947390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41457684-37a1-4455-b378-3093950686c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:51.949905 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41457684-37a1-4455-b378-3093950686c5 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:51.967252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:52.075027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1397/2780] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:17:51 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:52.079681 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba8995-edd2-4c53-ac02-80262078af41 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:52.091520 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba8995-edd2-4c53-ac02-80262078af41 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:52.151806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d847828-666f-496f-bd78-32e5ec41e531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:52.152599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d847828-666f-496f-bd78-32e5ec41e531 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:52.152995 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d847828-666f-496f-bd78-32e5ec41e531 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:52.153244 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d847828-666f-496f-bd78-32e5ec41e531 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:52.153936 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d847828-666f-496f-bd78-32e5ec41e531 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:52.156071 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d847828-666f-496f-bd78-32e5ec41e531 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:52.164524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:52.288809 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1384/2781] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:17:52 2022] POST /identity/v3/users => generated 287 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:17:52.294382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41457684-37a1-4455-b378-3093950686c5 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:52.305143 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41457684-37a1-4455-b378-3093950686c5 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:52.310214 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3e2e9-87ff-437c-86e5-8d4242d465b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:52.310852 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3e2e9-87ff-437c-86e5-8d4242d465b4 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:52.311248 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3e2e9-87ff-437c-86e5-8d4242d465b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:52.311425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3e2e9-87ff-437c-86e5-8d4242d465b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:52.312260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3e2e9-87ff-437c-86e5-8d4242d465b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:52.313843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3e2e9-87ff-437c-86e5-8d4242d465b4 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:52.325714 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1398/2782] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:17:52.331378 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d847828-666f-496f-bd78-32e5ec41e531 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:52.342598 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d847828-666f-496f-bd78-32e5ec41e531 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:52.347417 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f0888-7f9f-4e64-822b-5c1af1647d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:52.348061 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0888-7f9f-4e64-822b-5c1af1647d18 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:52.348350 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0888-7f9f-4e64-822b-5c1af1647d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:52.348623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0888-7f9f-4e64-822b-5c1af1647d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:52.349326 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f0888-7f9f-4e64-822b-5c1af1647d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:52.350918 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f0888-7f9f-4e64-822b-5c1af1647d18 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:52.364099 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1385/2783] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:17:52.369059 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3e2e9-87ff-437c-86e5-8d4242d465b4 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:52.379576 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3e2e9-87ff-437c-86e5-8d4242d465b4 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:52.384233 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c496ee-f59c-4ea6-8539-9e527b7c2684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:52.384890 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c496ee-f59c-4ea6-8539-9e527b7c2684 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:52.385170 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c496ee-f59c-4ea6-8539-9e527b7c2684 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:52.385444 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c496ee-f59c-4ea6-8539-9e527b7c2684 admin admin] PATH_INFO: `/v3/projects/e01388e7ebf04902a6feb4120567dada/users/d74c7f0d6697402b82bb7c0034702f27/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:52.404174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c496ee-f59c-4ea6-8539-9e527b7c2684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01388e7ebf04902a6feb4120567dada, user_id=d74c7f0d6697402b82bb7c0034702f27, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:52.406229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c496ee-f59c-4ea6-8539-9e527b7c2684 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:52.502872 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f0888-7f9f-4e64-822b-5c1af1647d18 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:52.504624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-795f0888-7f9f-4e64-822b-5c1af1647d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:52.524115 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1399/2784] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:17:52 2022] PUT /identity/v3/projects/e01388e7ebf04902a6feb4120567dada/users/d74c7f0d6697402b82bb7c0034702f27/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:17:52.535745 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c3c44-db5e-445f-94bf-cd2038ccc81d None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:52.536008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c3c44-db5e-445f-94bf-cd2038ccc81d None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:52.536289 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c3c44-db5e-445f-94bf-cd2038ccc81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:52.566112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-719c3c44-db5e-445f-94bf-cd2038ccc81d None None] MFA Rules not processed for user `d74c7f0d6697402b82bb7c0034702f27`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:52.567230 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-719c3c44-db5e-445f-94bf-cd2038ccc81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:52.582490 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f0888-7f9f-4e64-822b-5c1af1647d18 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:52.635003 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1400/2785] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:17:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:17:52.640654 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c3c44-db5e-445f-94bf-cd2038ccc81d None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:52.655969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9490b3-ec2f-4209-a621-f6cb7e87448e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a0728e2f6249bdb45d6f44fb15aa4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:52.656798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9490b3-ec2f-4209-a621-f6cb7e87448e service placement] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:52.657190 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9490b3-ec2f-4209-a621-f6cb7e87448e service placement] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:52.657641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9490b3-ec2f-4209-a621-f6cb7e87448e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:52.659904 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-7a9490b3-ec2f-4209-a621-f6cb7e87448e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:52.672995 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9490b3-ec2f-4209-a621-f6cb7e87448e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:52.674608 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9490b3-ec2f-4209-a621-f6cb7e87448e service placement] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:52.686093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1386/2786] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:17:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:17:52.740556 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c3c44-db5e-445f-94bf-cd2038ccc81d None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:52.748292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c1f0a-064e-4451-91da-092b45e720ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:52.749283 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c1f0a-064e-4451-91da-092b45e720ec admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:52.749809 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c1f0a-064e-4451-91da-092b45e720ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:52.750282 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c1f0a-064e-4451-91da-092b45e720ec admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:52.751436 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c1f0a-064e-4451-91da-092b45e720ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:52.754979 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c1f0a-064e-4451-91da-092b45e720ec admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:52.900414 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1401/2787] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:17:52 2022] POST /identity/v3/projects => generated 371 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:52.905274 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9490b3-ec2f-4209-a621-f6cb7e87448e service placement] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:52.916531 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9490b3-ec2f-4209-a621-f6cb7e87448e service placement] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:52.980499 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7ae62-aabc-467e-ba50-75328a9d2ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:52.981128 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7ae62-aabc-467e-ba50-75328a9d2ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:52.981401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7ae62-aabc-467e-ba50-75328a9d2ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:52.981672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7ae62-aabc-467e-ba50-75328a9d2ad9 admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:52.982337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7ae62-aabc-467e-ba50-75328a9d2ad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:52.984258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7ae62-aabc-467e-ba50-75328a9d2ad9 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:53.091449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:53.096859 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1387/2788] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:17:52 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:53.100907 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c1f0a-064e-4451-91da-092b45e720ec admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:53.112017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c1f0a-064e-4451-91da-092b45e720ec admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:53.116949 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f3399-5e17-4741-a03d-b3417f0e794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:53.117691 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f3399-5e17-4741-a03d-b3417f0e794d admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:53.118013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f3399-5e17-4741-a03d-b3417f0e794d admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:53.118325 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f3399-5e17-4741-a03d-b3417f0e794d admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:53.119140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f3399-5e17-4741-a03d-b3417f0e794d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:53.121376 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f3399-5e17-4741-a03d-b3417f0e794d admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:53.131808 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1402/2789] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:17:53.139720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7ae62-aabc-467e-ba50-75328a9d2ad9 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:53.150369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7ae62-aabc-467e-ba50-75328a9d2ad9 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:53.167157 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26939c56-e4c3-4c23-91b1-440906c2cc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:53.167834 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26939c56-e4c3-4c23-91b1-440906c2cc3b admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:53.168114 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26939c56-e4c3-4c23-91b1-440906c2cc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:53.168386 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26939c56-e4c3-4c23-91b1-440906c2cc3b admin admin] PATH_INFO: `/v3/projects/fa4e156c67574ff6ad6e26a3f07e06c6/users/0c82c93d40da455c904495852b2959dd/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:53.185203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26939c56-e4c3-4c23-91b1-440906c2cc3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4e156c67574ff6ad6e26a3f07e06c6, user_id=0c82c93d40da455c904495852b2959dd, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:53.187887 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26939c56-e4c3-4c23-91b1-440906c2cc3b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:53.262325 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1388/2790] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:17:53 2022] PUT /identity/v3/projects/fa4e156c67574ff6ad6e26a3f07e06c6/users/0c82c93d40da455c904495852b2959dd/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:17:53.266749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f3399-5e17-4741-a03d-b3417f0e794d admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:53.277916 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f3399-5e17-4741-a03d-b3417f0e794d admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:53.302134 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:53.342339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc38788-b6c8-43d3-915e-0c393515e312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:53.342810 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc38788-b6c8-43d3-915e-0c393515e312 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:53.343090 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc38788-b6c8-43d3-915e-0c393515e312 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:53.343371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc38788-b6c8-43d3-915e-0c393515e312 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:53.344097 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc38788-b6c8-43d3-915e-0c393515e312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:53.346323 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc38788-b6c8-43d3-915e-0c393515e312 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:53.357619 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1403/2791] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:17:53.362372 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26939c56-e4c3-4c23-91b1-440906c2cc3b admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:53.374322 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26939c56-e4c3-4c23-91b1-440906c2cc3b admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:53.379020 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d611-efd4-4bad-8399-507cc3ed5b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:53.384279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d611-efd4-4bad-8399-507cc3ed5b0a admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:53.384279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d611-efd4-4bad-8399-507cc3ed5b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:53.384279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d611-efd4-4bad-8399-507cc3ed5b0a admin admin] PATH_INFO: `/v3/projects/fa4e156c67574ff6ad6e26a3f07e06c6/users/0c82c93d40da455c904495852b2959dd/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:53.384279 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d611-efd4-4bad-8399-507cc3ed5b0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4e156c67574ff6ad6e26a3f07e06c6, user_id=0c82c93d40da455c904495852b2959dd, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:53.385092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d611-efd4-4bad-8399-507cc3ed5b0a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:53.397515 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1389/2792] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:17:53 2022] PUT /identity/v3/projects/fa4e156c67574ff6ad6e26a3f07e06c6/users/0c82c93d40da455c904495852b2959dd/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:17:53.410103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbb4d6-9dc7-4b6e-bb37-7f856fe32d5f None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:53.410530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbb4d6-9dc7-4b6e-bb37-7f856fe32d5f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:53.410908 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbb4d6-9dc7-4b6e-bb37-7f856fe32d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:53.454782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-dafbb4d6-9dc7-4b6e-bb37-7f856fe32d5f None None] MFA Rules not processed for user `0c82c93d40da455c904495852b2959dd`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:53.465233 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-dafbb4d6-9dc7-4b6e-bb37-7f856fe32d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:53.532229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1404/2793] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:17:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:17:53.544675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d611-efd4-4bad-8399-507cc3ed5b0a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:53.546418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3b35d611-efd4-4bad-8399-507cc3ed5b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:53.609369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d611-efd4-4bad-8399-507cc3ed5b0a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:53.613072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f089ac-24a9-4681-ad8e-e3207fc8ff93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:53.613694 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f089ac-24a9-4681-ad8e-e3207fc8ff93 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:53.613976 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f089ac-24a9-4681-ad8e-e3207fc8ff93 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:53.614251 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f089ac-24a9-4681-ad8e-e3207fc8ff93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:53.624228 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f089ac-24a9-4681-ad8e-e3207fc8ff93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:53.625648 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f089ac-24a9-4681-ad8e-e3207fc8ff93 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:53.636457 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1390/2794] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:17:53 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:17:54.793832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbb4d6-9dc7-4b6e-bb37-7f856fe32d5f None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:54.858542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbb4d6-9dc7-4b6e-bb37-7f856fe32d5f None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:54.863187 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b981d-0ca2-43b4-a5a6-7404ca4d085f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:54.863860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b981d-0ca2-43b4-a5a6-7404ca4d085f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:54.864138 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b981d-0ca2-43b4-a5a6-7404ca4d085f admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:54.864467 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b981d-0ca2-43b4-a5a6-7404ca4d085f admin admin] PATH_INFO: `/v3/users/d74c7f0d6697402b82bb7c0034702f27` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:54.865622 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b981d-0ca2-43b4-a5a6-7404ca4d085f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d74c7f0d6697402b82bb7c0034702f27)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:54.867545 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b981d-0ca2-43b4-a5a6-7404ca4d085f admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:55.104235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b96b981d-0ca2-43b4-a5a6-7404ca4d085f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd74c7f0d6697402b82bb7c0034702f27'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:55.106017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:55.112386 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b96b981d-0ca2-43b4-a5a6-7404ca4d085f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd74c7f0d6697402b82bb7c0034702f27'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:55.192499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b96b981d-0ca2-43b4-a5a6-7404ca4d085f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd74c7f0d6697402b82bb7c0034702f27'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:55.206439 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b96b981d-0ca2-43b4-a5a6-7404ca4d085f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd74c7f0d6697402b82bb7c0034702f27'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:55.207165 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-b96b981d-0ca2-43b4-a5a6-7404ca4d085f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd74c7f0d6697402b82bb7c0034702f27'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:55.213713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1405/2795] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:17:54 2022] DELETE /identity/v3/users/d74c7f0d6697402b82bb7c0034702f27 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:17:55.218816 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e764dd-5faf-47d2-8789-123b00b62d43 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:55.219307 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e764dd-5faf-47d2-8789-123b00b62d43 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:55.219618 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e764dd-5faf-47d2-8789-123b00b62d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:55.241052 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-f2e764dd-5faf-47d2-8789-123b00b62d43 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:55.247039 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f2e764dd-5faf-47d2-8789-123b00b62d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:55.312409 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1391/2796] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:17:55.322263 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b981d-0ca2-43b4-a5a6-7404ca4d085f admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:55.324775 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b96b981d-0ca2-43b4-a5a6-7404ca4d085f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:55.383806 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b981d-0ca2-43b4-a5a6-7404ca4d085f admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:55.388514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476e16a-75c0-4662-9e5d-564cf84cc75d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:55.389111 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476e16a-75c0-4662-9e5d-564cf84cc75d service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:55.389390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476e16a-75c0-4662-9e5d-564cf84cc75d service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:55.389656 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476e16a-75c0-4662-9e5d-564cf84cc75d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:55.399016 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476e16a-75c0-4662-9e5d-564cf84cc75d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:55.402480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476e16a-75c0-4662-9e5d-564cf84cc75d service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:55.415838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1406/2797] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:17:55.870299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e764dd-5faf-47d2-8789-123b00b62d43 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:55.872014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f2e764dd-5faf-47d2-8789-123b00b62d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:55.885623 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e764dd-5faf-47d2-8789-123b00b62d43 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:55.891440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:55.892184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:55.892466 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:55.892744 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] PATH_INFO: `/v3/projects/e01388e7ebf04902a6feb4120567dada` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:55.893966 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01388e7ebf04902a6feb4120567dada)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:55.895944 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:56.084865 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01388e7ebf04902a6feb4120567dada', 'request_id': 'req-931e2525-4bdc-495b-8243-9b5c2e5b3281'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:56.092716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01388e7ebf04902a6feb4120567dada', 'request_id': 'req-931e2525-4bdc-495b-8243-9b5c2e5b3281'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:56.097479 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:56.099191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01388e7ebf04902a6feb4120567dada', 'request_id': 'req-931e2525-4bdc-495b-8243-9b5c2e5b3281'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:56.193979 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] The token cache is being invalidate because project e01388e7ebf04902a6feb4120567dada was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:17:56.194373 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:56.195492 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:56.198578 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1392/2798] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:17:55 2022] DELETE /identity/v3/projects/e01388e7ebf04902a6feb4120567dada => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:17:56.203556 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476e16a-75c0-4662-9e5d-564cf84cc75d service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:56.215358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476e16a-75c0-4662-9e5d-564cf84cc75d service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:56.286441 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3232ed0-5013-4ff0-bce5-20564e6752a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:56.287118 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3232ed0-5013-4ff0-bce5-20564e6752a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:56.287395 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3232ed0-5013-4ff0-bce5-20564e6752a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:56.287700 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3232ed0-5013-4ff0-bce5-20564e6752a9 admin admin] PATH_INFO: `/v3/users/0c82c93d40da455c904495852b2959dd` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:56.288869 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3232ed0-5013-4ff0-bce5-20564e6752a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c82c93d40da455c904495852b2959dd)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:56.290836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3232ed0-5013-4ff0-bce5-20564e6752a9 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:56.594893 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a3232ed0-5013-4ff0-bce5-20564e6752a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c82c93d40da455c904495852b2959dd'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:56.601911 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a3232ed0-5013-4ff0-bce5-20564e6752a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c82c93d40da455c904495852b2959dd'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:56.712439 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a3232ed0-5013-4ff0-bce5-20564e6752a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c82c93d40da455c904495852b2959dd'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:56.718627 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:56.727738 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a3232ed0-5013-4ff0-bce5-20564e6752a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c82c93d40da455c904495852b2959dd'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:56.728550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-a3232ed0-5013-4ff0-bce5-20564e6752a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c82c93d40da455c904495852b2959dd'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:56.736838 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1407/2799] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:17:56 2022] DELETE /identity/v3/users/0c82c93d40da455c904495852b2959dd => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:17:57.244082 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:57.246977 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:57.264084 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e2525-4bdc-495b-8243-9b5c2e5b3281 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:57.335077 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197a791-fa5d-4e5c-889a-147bb28c5987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:57.335760 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197a791-fa5d-4e5c-889a-147bb28c5987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:57.336049 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197a791-fa5d-4e5c-889a-147bb28c5987 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:57.336329 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197a791-fa5d-4e5c-889a-147bb28c5987 admin admin] PATH_INFO: `/v3/projects/fa4e156c67574ff6ad6e26a3f07e06c6` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:57.337533 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197a791-fa5d-4e5c-889a-147bb28c5987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4e156c67574ff6ad6e26a3f07e06c6)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:57.340174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197a791-fa5d-4e5c-889a-147bb28c5987 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:57.452949 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a197a791-fa5d-4e5c-889a-147bb28c5987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4e156c67574ff6ad6e26a3f07e06c6', 'request_id': 'req-a197a791-fa5d-4e5c-889a-147bb28c5987'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:57.459150 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a197a791-fa5d-4e5c-889a-147bb28c5987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4e156c67574ff6ad6e26a3f07e06c6', 'request_id': 'req-a197a791-fa5d-4e5c-889a-147bb28c5987'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:57.465561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a197a791-fa5d-4e5c-889a-147bb28c5987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4e156c67574ff6ad6e26a3f07e06c6', 'request_id': 'req-a197a791-fa5d-4e5c-889a-147bb28c5987'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:57.542493 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a197a791-fa5d-4e5c-889a-147bb28c5987 admin admin] The token cache is being invalidate because project fa4e156c67574ff6ad6e26a3f07e06c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:17:57.542822 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a197a791-fa5d-4e5c-889a-147bb28c5987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:57.543523 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a197a791-fa5d-4e5c-889a-147bb28c5987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:17:57.545241 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1393/2800] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:17:57 2022] DELETE /identity/v3/projects/fa4e156c67574ff6ad6e26a3f07e06c6 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:17:57.551124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc10c31-58db-4f97-a04f-112a4bb26148 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:57.551834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc10c31-58db-4f97-a04f-112a4bb26148 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:57.551834 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc10c31-58db-4f97-a04f-112a4bb26148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:57.572840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-7dc10c31-58db-4f97-a04f-112a4bb26148 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:57.574158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7dc10c31-58db-4f97-a04f-112a4bb26148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:57.640469 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1408/2801] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:17:57.648166 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208b890-1391-4cc4-af36-1ac2663dd3f1 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:57.648455 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208b890-1391-4cc4-af36-1ac2663dd3f1 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:57.648736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208b890-1391-4cc4-af36-1ac2663dd3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:57.672729 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-d208b890-1391-4cc4-af36-1ac2663dd3f1 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:57.673970 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-d208b890-1391-4cc4-af36-1ac2663dd3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:57.682399 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1394/2802] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:17:57.686913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc10c31-58db-4f97-a04f-112a4bb26148 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:57.696860 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc10c31-58db-4f97-a04f-112a4bb26148 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:57.703199 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62821529-0525-4597-9738-a32b0f70b195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:57.703843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62821529-0525-4597-9738-a32b0f70b195 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:57.704116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62821529-0525-4597-9738-a32b0f70b195 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:57.704387 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62821529-0525-4597-9738-a32b0f70b195 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:57.705088 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62821529-0525-4597-9738-a32b0f70b195 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:17:57.705390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62821529-0525-4597-9738-a32b0f70b195 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:57.707004 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62821529-0525-4597-9738-a32b0f70b195 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:57.720010 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1409/2803] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:17:57.724601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208b890-1391-4cc4-af36-1ac2663dd3f1 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:57.734934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208b890-1391-4cc4-af36-1ac2663dd3f1 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:57.739969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0077b05-cf1b-4b2f-9113-76f9a5942061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:57.740593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0077b05-cf1b-4b2f-9113-76f9a5942061 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:57.740870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0077b05-cf1b-4b2f-9113-76f9a5942061 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:57.741150 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0077b05-cf1b-4b2f-9113-76f9a5942061 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:57.741824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0077b05-cf1b-4b2f-9113-76f9a5942061 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:57.743757 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0077b05-cf1b-4b2f-9113-76f9a5942061 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:57.868315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1395/2804] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:17:57 2022] POST /identity/v3/projects => generated 407 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:17:57.872727 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62821529-0525-4597-9738-a32b0f70b195 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:57.883388 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62821529-0525-4597-9738-a32b0f70b195 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:57.944371 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865550bd-4e54-4e10-a2b0-778a6d0129b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:57.945015 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865550bd-4e54-4e10-a2b0-778a6d0129b0 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:57.945299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865550bd-4e54-4e10-a2b0-778a6d0129b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:57.945567 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865550bd-4e54-4e10-a2b0-778a6d0129b0 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:57.946233 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865550bd-4e54-4e10-a2b0-778a6d0129b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:57.948797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865550bd-4e54-4e10-a2b0-778a6d0129b0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:58.066043 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1410/2805] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:17:57 2022] POST /identity/v3/users => generated 304 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:58.072114 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0077b05-cf1b-4b2f-9113-76f9a5942061 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:58.085614 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0077b05-cf1b-4b2f-9113-76f9a5942061 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:58.091114 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed8933-1de0-4c1c-bd55-8f111e9451b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:58.091761 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed8933-1de0-4c1c-bd55-8f111e9451b6 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:58.092124 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed8933-1de0-4c1c-bd55-8f111e9451b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:58.092441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed8933-1de0-4c1c-bd55-8f111e9451b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:58.093161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed8933-1de0-4c1c-bd55-8f111e9451b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:58.094745 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed8933-1de0-4c1c-bd55-8f111e9451b6 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:58.105914 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1396/2806] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:17:58.110675 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865550bd-4e54-4e10-a2b0-778a6d0129b0 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:58.122331 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865550bd-4e54-4e10-a2b0-778a6d0129b0 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:58.131146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f60d4-ce87-4d9b-9f7a-a7333351dc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:58.132594 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f60d4-ce87-4d9b-9f7a-a7333351dc97 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:58.132594 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f60d4-ce87-4d9b-9f7a-a7333351dc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:58.132594 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f60d4-ce87-4d9b-9f7a-a7333351dc97 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:58.133049 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f60d4-ce87-4d9b-9f7a-a7333351dc97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:58.134671 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f60d4-ce87-4d9b-9f7a-a7333351dc97 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:58.147883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1411/2807] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:17:58.152481 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed8933-1de0-4c1c-bd55-8f111e9451b6 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:58.163037 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed8933-1de0-4c1c-bd55-8f111e9451b6 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:58.168411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d8c80-5db5-45c7-8c5d-a28cd5fcd6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:58.169074 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d8c80-5db5-45c7-8c5d-a28cd5fcd6a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:58.169351 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d8c80-5db5-45c7-8c5d-a28cd5fcd6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:58.169627 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d8c80-5db5-45c7-8c5d-a28cd5fcd6a7 admin admin] PATH_INFO: `/v3/projects/08c2b580445545b2860a2fadce204f06/users/f90ce240842144c8a1f4328d971a3e6d/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:58.185468 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d8c80-5db5-45c7-8c5d-a28cd5fcd6a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c2b580445545b2860a2fadce204f06, user_id=f90ce240842144c8a1f4328d971a3e6d, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:58.188137 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d8c80-5db5-45c7-8c5d-a28cd5fcd6a7 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:58.310056 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1397/2808] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:17:58 2022] PUT /identity/v3/projects/08c2b580445545b2860a2fadce204f06/users/f90ce240842144c8a1f4328d971a3e6d/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:17:58.322394 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fa846-1abc-427b-b613-a0249c228bfb None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:58.322798 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fa846-1abc-427b-b613-a0249c228bfb None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:58.323773 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fa846-1abc-427b-b613-a0249c228bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:58.362343 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-086fa846-1abc-427b-b613-a0249c228bfb None None] MFA Rules not processed for user `f90ce240842144c8a1f4328d971a3e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:58.365800 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-086fa846-1abc-427b-b613-a0249c228bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:58.414962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:58.441640 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1412/2809] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:17:58 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:17:58.446716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d8c80-5db5-45c7-8c5d-a28cd5fcd6a7 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:58.467692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:58.517131 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d8c80-5db5-45c7-8c5d-a28cd5fcd6a7 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:58.523957 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3df718-6fe6-4745-91c8-0ceb4bf92165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:58.526033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3df718-6fe6-4745-91c8-0ceb4bf92165 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:58.526033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3df718-6fe6-4745-91c8-0ceb4bf92165 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:58.526033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3df718-6fe6-4745-91c8-0ceb4bf92165 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:58.526597 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3df718-6fe6-4745-91c8-0ceb4bf92165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:58.528436 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3df718-6fe6-4745-91c8-0ceb4bf92165 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:58.628418 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1398/2810] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:17:58 2022] POST /identity/v3/projects => generated 407 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:58.632918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fa846-1abc-427b-b613-a0249c228bfb None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:58.645306 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fa846-1abc-427b-b613-a0249c228bfb None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:58.704914 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5593f-3adf-4d75-8d9c-4337ad59607e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:58.705466 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5593f-3adf-4d75-8d9c-4337ad59607e admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:58.705747 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5593f-3adf-4d75-8d9c-4337ad59607e admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:58.706023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5593f-3adf-4d75-8d9c-4337ad59607e admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:58.706699 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5593f-3adf-4d75-8d9c-4337ad59607e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:58.708654 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5593f-3adf-4d75-8d9c-4337ad59607e admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:58.839075 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1413/2811] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:17:58 2022] POST /identity/v3/users => generated 310 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:17:58.844121 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3df718-6fe6-4745-91c8-0ceb4bf92165 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:58.856345 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3df718-6fe6-4745-91c8-0ceb4bf92165 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:58.861088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbbe0c-5399-4635-accc-db2f2f0f8435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:58.861701 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbbe0c-5399-4635-accc-db2f2f0f8435 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:58.861988 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbbe0c-5399-4635-accc-db2f2f0f8435 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:58.862267 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbbe0c-5399-4635-accc-db2f2f0f8435 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:58.862970 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbbe0c-5399-4635-accc-db2f2f0f8435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:58.864676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbbe0c-5399-4635-accc-db2f2f0f8435 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:58.878142 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1399/2812] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:17:58.882492 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5593f-3adf-4d75-8d9c-4337ad59607e admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:58.893193 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5593f-3adf-4d75-8d9c-4337ad59607e admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:58.898429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c168b-d60e-4df7-8f13-c23ca23f3d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:58.899108 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c168b-d60e-4df7-8f13-c23ca23f3d4a admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:58.899383 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c168b-d60e-4df7-8f13-c23ca23f3d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:58.899692 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c168b-d60e-4df7-8f13-c23ca23f3d4a admin admin] PATH_INFO: `/v3/projects/0c1ee4aae3034483ab7b27e53c42b247/users/2e6ac985e8b4447bae8b4aab8b8f0194/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:58.919832 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c168b-d60e-4df7-8f13-c23ca23f3d4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c1ee4aae3034483ab7b27e53c42b247, user_id=2e6ac985e8b4447bae8b4aab8b8f0194, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:58.922881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c168b-d60e-4df7-8f13-c23ca23f3d4a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:59.034140 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1414/2813] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:17:58 2022] PUT /identity/v3/projects/0c1ee4aae3034483ab7b27e53c42b247/users/2e6ac985e8b4447bae8b4aab8b8f0194/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:17:59.038364 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbbe0c-5399-4635-accc-db2f2f0f8435 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:59.050227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbbe0c-5399-4635-accc-db2f2f0f8435 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:59.113870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a4a17-4dc9-4864-b56b-bf37775e2da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:59.114628 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a4a17-4dc9-4864-b56b-bf37775e2da2 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:59.115009 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a4a17-4dc9-4864-b56b-bf37775e2da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:59.115508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a4a17-4dc9-4864-b56b-bf37775e2da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:59.116403 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a4a17-4dc9-4864-b56b-bf37775e2da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:59.118103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a4a17-4dc9-4864-b56b-bf37775e2da2 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:59.128827 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1400/2814] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 15:17:59.133598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c168b-d60e-4df7-8f13-c23ca23f3d4a admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:59.144634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c168b-d60e-4df7-8f13-c23ca23f3d4a admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:59.149390 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c715e-2fb6-4d1d-bd38-c95d7035ed65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:59.150053 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c715e-2fb6-4d1d-bd38-c95d7035ed65 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:59.150382 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c715e-2fb6-4d1d-bd38-c95d7035ed65 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:59.150651 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c715e-2fb6-4d1d-bd38-c95d7035ed65 admin admin] PATH_INFO: `/v3/projects/0c1ee4aae3034483ab7b27e53c42b247/users/2e6ac985e8b4447bae8b4aab8b8f0194/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:59.153226 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c715e-2fb6-4d1d-bd38-c95d7035ed65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c1ee4aae3034483ab7b27e53c42b247, user_id=2e6ac985e8b4447bae8b4aab8b8f0194, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:59.155918 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c715e-2fb6-4d1d-bd38-c95d7035ed65 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:59.168137 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1415/2815] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:17:59 2022] PUT /identity/v3/projects/0c1ee4aae3034483ab7b27e53c42b247/users/2e6ac985e8b4447bae8b4aab8b8f0194/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:17:59.180838 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b98de3-df49-4b27-b874-3c22e8bc7ac5 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:59.181132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b98de3-df49-4b27-b874-3c22e8bc7ac5 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:59.181390 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b98de3-df49-4b27-b874-3c22e8bc7ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:59.213149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-99b98de3-df49-4b27-b874-3c22e8bc7ac5 None None] MFA Rules not processed for user `2e6ac985e8b4447bae8b4aab8b8f0194`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:17:59.214953 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-99b98de3-df49-4b27-b874-3c22e8bc7ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:59.289521 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1401/2816] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:17:59 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:17:59.301870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c715e-2fb6-4d1d-bd38-c95d7035ed65 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:59.303787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c84c715e-2fb6-4d1d-bd38-c95d7035ed65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:17:59.366642 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c715e-2fb6-4d1d-bd38-c95d7035ed65 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:59.372065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d36ec1-b678-4aec-ac18-f0b7827a0467 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:59.372697 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d36ec1-b678-4aec-ac18-f0b7827a0467 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:59.372985 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d36ec1-b678-4aec-ac18-f0b7827a0467 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:59.373255 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d36ec1-b678-4aec-ac18-f0b7827a0467 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:59.383304 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d36ec1-b678-4aec-ac18-f0b7827a0467 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:59.385358 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d36ec1-b678-4aec-ac18-f0b7827a0467 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:59.396434 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1416/2817] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:17:59 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:17:59.823629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b98de3-df49-4b27-b874-3c22e8bc7ac5 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:17:59.833788 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b98de3-df49-4b27-b874-3c22e8bc7ac5 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:17:59.837884 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8022d-3284-4818-891c-60caf654d93c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:17:59.838698 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8022d-3284-4818-891c-60caf654d93c service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:17:59.839100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8022d-3284-4818-891c-60caf654d93c service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:17:59.839553 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8022d-3284-4818-891c-60caf654d93c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:17:59.849604 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:17:59.900516 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8022d-3284-4818-891c-60caf654d93c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:17:59.902772 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8022d-3284-4818-891c-60caf654d93c service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:17:59.913973 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1402/2818] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:17:59 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:18:09.391541 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:10.037679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=23 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:10.432858 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d36ec1-b678-4aec-ac18-f0b7827a0467 service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:10.501132 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d36ec1-b678-4aec-ac18-f0b7827a0467 service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:10.505783 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d8e0f-b871-48fe-8121-b2c193061298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:10.506421 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d8e0f-b871-48fe-8121-b2c193061298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:10.506702 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d8e0f-b871-48fe-8121-b2c193061298 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:10.506977 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d8e0f-b871-48fe-8121-b2c193061298 admin admin] PATH_INFO: `/v3/users/f90ce240842144c8a1f4328d971a3e6d` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:10.508177 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d8e0f-b871-48fe-8121-b2c193061298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f90ce240842144c8a1f4328d971a3e6d)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:10.510120 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d8e0f-b871-48fe-8121-b2c193061298 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:10.739439 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-864d8e0f-b871-48fe-8121-b2c193061298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f90ce240842144c8a1f4328d971a3e6d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:10.747350 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-864d8e0f-b871-48fe-8121-b2c193061298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f90ce240842144c8a1f4328d971a3e6d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:10.860942 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-864d8e0f-b871-48fe-8121-b2c193061298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f90ce240842144c8a1f4328d971a3e6d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:10.875074 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-864d8e0f-b871-48fe-8121-b2c193061298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f90ce240842144c8a1f4328d971a3e6d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:10.875843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-864d8e0f-b871-48fe-8121-b2c193061298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f90ce240842144c8a1f4328d971a3e6d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:10.882036 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1417/2819] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:18:10 2022] DELETE /identity/v3/users/f90ce240842144c8a1f4328d971a3e6d => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:10.886752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ce397-086b-40e1-9bdf-9ce7303ca1ff None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:10.887042 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ce397-086b-40e1-9bdf-9ce7303ca1ff None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:10.887323 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ce397-086b-40e1-9bdf-9ce7303ca1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:10.909092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-dd5ce397-086b-40e1-9bdf-9ce7303ca1ff None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:10.910361 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-dd5ce397-086b-40e1-9bdf-9ce7303ca1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:10.978674 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1403/2820] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:18:10.989835 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d8e0f-b871-48fe-8121-b2c193061298 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:10.991557 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-864d8e0f-b871-48fe-8121-b2c193061298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:11.052398 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d8e0f-b871-48fe-8121-b2c193061298 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:11.056020 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad245280-445b-4e59-a102-caa195a3bd78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:11.056662 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad245280-445b-4e59-a102-caa195a3bd78 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:11.056969 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad245280-445b-4e59-a102-caa195a3bd78 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:11.057245 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad245280-445b-4e59-a102-caa195a3bd78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:11.067681 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad245280-445b-4e59-a102-caa195a3bd78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:11.069219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad245280-445b-4e59-a102-caa195a3bd78 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:11.080657 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1418/2821] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:18:11.605755 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ce397-086b-40e1-9bdf-9ce7303ca1ff None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:11.607323 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-dd5ce397-086b-40e1-9bdf-9ce7303ca1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:11.620115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ce397-086b-40e1-9bdf-9ce7303ca1ff None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:11.625510 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:11.626704 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:11.626704 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:11.626704 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] PATH_INFO: `/v3/projects/08c2b580445545b2860a2fadce204f06` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:11.627938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c2b580445545b2860a2fadce204f06)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:11.629844 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:11.785938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c2b580445545b2860a2fadce204f06', 'request_id': 'req-e0e3c822-c2d1-4182-9896-32711731a8e7'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:11.792243 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c2b580445545b2860a2fadce204f06', 'request_id': 'req-e0e3c822-c2d1-4182-9896-32711731a8e7'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:11.798127 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c2b580445545b2860a2fadce204f06', 'request_id': 'req-e0e3c822-c2d1-4182-9896-32711731a8e7'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:11.917063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] The token cache is being invalidate because project 08c2b580445545b2860a2fadce204f06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:18:11.917282 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:11.918645 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:11.920249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1404/2822] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:18:11 2022] DELETE /identity/v3/projects/08c2b580445545b2860a2fadce204f06 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:18:11.924461 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad245280-445b-4e59-a102-caa195a3bd78 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:11.935194 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad245280-445b-4e59-a102-caa195a3bd78 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:11.953338 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:11.991333 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebd8bf-941a-4951-9fd0-eeb2af68f82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:11.992006 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd8bf-941a-4951-9fd0-eeb2af68f82c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:11.992291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd8bf-941a-4951-9fd0-eeb2af68f82c admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:11.992568 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd8bf-941a-4951-9fd0-eeb2af68f82c admin admin] PATH_INFO: `/v3/users/2e6ac985e8b4447bae8b4aab8b8f0194` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:11.993749 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebd8bf-941a-4951-9fd0-eeb2af68f82c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e6ac985e8b4447bae8b4aab8b8f0194)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:11.995690 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebd8bf-941a-4951-9fd0-eeb2af68f82c admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:12.264609 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-47ebd8bf-941a-4951-9fd0-eeb2af68f82c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e6ac985e8b4447bae8b4aab8b8f0194'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:12.272030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-47ebd8bf-941a-4951-9fd0-eeb2af68f82c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e6ac985e8b4447bae8b4aab8b8f0194'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:12.369224 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-47ebd8bf-941a-4951-9fd0-eeb2af68f82c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e6ac985e8b4447bae8b4aab8b8f0194'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:12.382464 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-47ebd8bf-941a-4951-9fd0-eeb2af68f82c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e6ac985e8b4447bae8b4aab8b8f0194'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:12.383190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-47ebd8bf-941a-4951-9fd0-eeb2af68f82c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e6ac985e8b4447bae8b4aab8b8f0194'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:12.390074 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1419/2823] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:18:11 2022] DELETE /identity/v3/users/2e6ac985e8b4447bae8b4aab8b8f0194 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:12.857133 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:12.860504 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:12.874465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3c822-c2d1-4182-9896-32711731a8e7 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:12.935184 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:12.935856 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:12.936139 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:12.936412 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c admin admin] PATH_INFO: `/v3/projects/0c1ee4aae3034483ab7b27e53c42b247` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:12.937562 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c1ee4aae3034483ab7b27e53c42b247)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:12.939456 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:13.033214 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:13.049509 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c1ee4aae3034483ab7b27e53c42b247', 'request_id': 'req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:13.055823 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c1ee4aae3034483ab7b27e53c42b247', 'request_id': 'req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:13.062161 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c1ee4aae3034483ab7b27e53c42b247', 'request_id': 'req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:13.151571 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c admin admin] The token cache is being invalidate because project 0c1ee4aae3034483ab7b27e53c42b247 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:18:13.151795 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:13.152546 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e13fef97-8eb0-4d67-be5b-c427ae7a8e1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:13.154209 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1405/2824] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:18:12 2022] DELETE /identity/v3/projects/0c1ee4aae3034483ab7b27e53c42b247 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:13.159899 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfe7bd-dc18-4989-b9e1-024e1daac4b1 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:13.160402 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfe7bd-dc18-4989-b9e1-024e1daac4b1 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:13.160507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfe7bd-dc18-4989-b9e1-024e1daac4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:13.180425 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-bbbfe7bd-dc18-4989-b9e1-024e1daac4b1 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:13.181622 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-bbbfe7bd-dc18-4989-b9e1-024e1daac4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:13.252177 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1420/2825] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:18:13.260453 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78be9c-b49e-41a4-9b2e-c1e58f07712c None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:13.260740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78be9c-b49e-41a4-9b2e-c1e58f07712c None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:13.261024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78be9c-b49e-41a4-9b2e-c1e58f07712c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:13.288785 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-ef78be9c-b49e-41a4-9b2e-c1e58f07712c None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:13.290019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-ef78be9c-b49e-41a4-9b2e-c1e58f07712c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:13.298406 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1406/2826] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:18:13.310350 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfe7bd-dc18-4989-b9e1-024e1daac4b1 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:13.320719 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfe7bd-dc18-4989-b9e1-024e1daac4b1 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:13.326268 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ec58c-8d54-4138-ba01-ed3f69389809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:13.326874 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ec58c-8d54-4138-ba01-ed3f69389809 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:13.327148 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ec58c-8d54-4138-ba01-ed3f69389809 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:13.327415 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ec58c-8d54-4138-ba01-ed3f69389809 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:13.328127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ec58c-8d54-4138-ba01-ed3f69389809 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:18:13.328427 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ec58c-8d54-4138-ba01-ed3f69389809 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:13.330038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ec58c-8d54-4138-ba01-ed3f69389809 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:13.342558 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1421/2827] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:13.347287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78be9c-b49e-41a4-9b2e-c1e58f07712c None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:13.356971 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78be9c-b49e-41a4-9b2e-c1e58f07712c None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:13.361732 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf6dc8-d590-4534-824f-9bf58a9437e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:13.362392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf6dc8-d590-4534-824f-9bf58a9437e1 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:13.362692 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf6dc8-d590-4534-824f-9bf58a9437e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:13.363170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf6dc8-d590-4534-824f-9bf58a9437e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:13.364300 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf6dc8-d590-4534-824f-9bf58a9437e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:13.366831 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf6dc8-d590-4534-824f-9bf58a9437e1 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:13.462096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1407/2828] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:18:13 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:18:13.466266 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ec58c-8d54-4138-ba01-ed3f69389809 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:13.476299 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ec58c-8d54-4138-ba01-ed3f69389809 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:13.507400 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:13.538635 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cabec7-90df-4a2e-8564-40b47cde81eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:13.539260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cabec7-90df-4a2e-8564-40b47cde81eb admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:13.539762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cabec7-90df-4a2e-8564-40b47cde81eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:13.540256 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cabec7-90df-4a2e-8564-40b47cde81eb admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:13.541460 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cabec7-90df-4a2e-8564-40b47cde81eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:13.543936 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cabec7-90df-4a2e-8564-40b47cde81eb admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:13.667166 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1422/2829] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:18:13 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:18:13.672249 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf6dc8-d590-4534-824f-9bf58a9437e1 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:13.683128 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf6dc8-d590-4534-824f-9bf58a9437e1 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:13.689369 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd491a-ab18-4411-a34d-012708bc2cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:13.690679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd491a-ab18-4411-a34d-012708bc2cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:13.690679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd491a-ab18-4411-a34d-012708bc2cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:13.690679 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd491a-ab18-4411-a34d-012708bc2cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:13.691939 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd491a-ab18-4411-a34d-012708bc2cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:13.694220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd491a-ab18-4411-a34d-012708bc2cd7 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:13.706547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1408/2830] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:18:13.710863 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cabec7-90df-4a2e-8564-40b47cde81eb admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:13.725879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cabec7-90df-4a2e-8564-40b47cde81eb admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:13.732264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc828c0e-3039-47f9-a6e7-cedc719704e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:13.732883 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc828c0e-3039-47f9-a6e7-cedc719704e2 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:13.733158 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc828c0e-3039-47f9-a6e7-cedc719704e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:13.733427 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc828c0e-3039-47f9-a6e7-cedc719704e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:13.734116 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc828c0e-3039-47f9-a6e7-cedc719704e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:13.735710 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc828c0e-3039-47f9-a6e7-cedc719704e2 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:13.746974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1423/2831] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:18:13.751985 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd491a-ab18-4411-a34d-012708bc2cd7 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:13.762829 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd491a-ab18-4411-a34d-012708bc2cd7 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:13.767622 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8cec2-4fbf-4653-9df1-efaa342f1c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:13.768287 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8cec2-4fbf-4653-9df1-efaa342f1c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:13.768569 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8cec2-4fbf-4653-9df1-efaa342f1c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:13.768846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8cec2-4fbf-4653-9df1-efaa342f1c0d admin admin] PATH_INFO: `/v3/projects/addb874bfc32490d97d0e15624ae8de8/users/bc32b6f30ce140509b7c3f48927eb1a5/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:13.784562 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8cec2-4fbf-4653-9df1-efaa342f1c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=addb874bfc32490d97d0e15624ae8de8, user_id=bc32b6f30ce140509b7c3f48927eb1a5, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:13.787199 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8cec2-4fbf-4653-9df1-efaa342f1c0d admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:13.866172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1409/2832] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:18:13 2022] PUT /identity/v3/projects/addb874bfc32490d97d0e15624ae8de8/users/bc32b6f30ce140509b7c3f48927eb1a5/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:18:13.876710 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953570f-4523-4614-aa2b-8e1b4a004f0e None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:13.883236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953570f-4523-4614-aa2b-8e1b4a004f0e None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:13.883236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953570f-4523-4614-aa2b-8e1b4a004f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:13.907455 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b953570f-4523-4614-aa2b-8e1b4a004f0e None None] MFA Rules not processed for user `bc32b6f30ce140509b7c3f48927eb1a5`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:13.908709 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b953570f-4523-4614-aa2b-8e1b4a004f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:13.972368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1424/2833] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:18:13.977033 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8cec2-4fbf-4653-9df1-efaa342f1c0d admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:14.045214 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8cec2-4fbf-4653-9df1-efaa342f1c0d admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:14.049954 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac68a8-553a-49f7-8e29-ff420023ef76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:14.050570 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac68a8-553a-49f7-8e29-ff420023ef76 admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:14.050855 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac68a8-553a-49f7-8e29-ff420023ef76 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:14.051132 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac68a8-553a-49f7-8e29-ff420023ef76 admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:14.051829 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac68a8-553a-49f7-8e29-ff420023ef76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:14.053716 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac68a8-553a-49f7-8e29-ff420023ef76 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:14.163352 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1410/2834] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:18:13 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:18:14.167375 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953570f-4523-4614-aa2b-8e1b4a004f0e None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:14.178836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953570f-4523-4614-aa2b-8e1b4a004f0e None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:14.235335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862cc1e-5455-48fe-b664-0348e3b9a3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:14.235888 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862cc1e-5455-48fe-b664-0348e3b9a3dd admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:14.236176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862cc1e-5455-48fe-b664-0348e3b9a3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:14.236443 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862cc1e-5455-48fe-b664-0348e3b9a3dd admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:14.237141 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b862cc1e-5455-48fe-b664-0348e3b9a3dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:14.239039 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b862cc1e-5455-48fe-b664-0348e3b9a3dd admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:14.387367 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1425/2835] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:18:14 2022] POST /identity/v3/users => generated 298 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:18:14.391537 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac68a8-553a-49f7-8e29-ff420023ef76 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:14.402647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac68a8-553a-49f7-8e29-ff420023ef76 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:14.407304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fab30-7283-4bc4-8f43-b22b0cda5aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:14.407950 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fab30-7283-4bc4-8f43-b22b0cda5aff admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:14.408238 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fab30-7283-4bc4-8f43-b22b0cda5aff admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:14.408518 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795fab30-7283-4bc4-8f43-b22b0cda5aff admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:14.409220 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795fab30-7283-4bc4-8f43-b22b0cda5aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:14.410861 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795fab30-7283-4bc4-8f43-b22b0cda5aff admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:14.421621 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1411/2836] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:18:14.426205 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862cc1e-5455-48fe-b664-0348e3b9a3dd admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:14.436993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862cc1e-5455-48fe-b664-0348e3b9a3dd admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:14.441725 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1935ee-f1c2-4d93-98af-e736c5865663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:14.442428 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1935ee-f1c2-4d93-98af-e736c5865663 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:14.442703 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1935ee-f1c2-4d93-98af-e736c5865663 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:14.442972 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1935ee-f1c2-4d93-98af-e736c5865663 admin admin] PATH_INFO: `/v3/projects/76f0f040846a4c5a91a8c76a15e0e1cb/users/6326fc96ca544c7283d384a13a13537a/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:14.458542 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1935ee-f1c2-4d93-98af-e736c5865663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76f0f040846a4c5a91a8c76a15e0e1cb, user_id=6326fc96ca544c7283d384a13a13537a, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:14.461281 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1935ee-f1c2-4d93-98af-e736c5865663 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:14.581632 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1426/2837] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:18:14 2022] PUT /identity/v3/projects/76f0f040846a4c5a91a8c76a15e0e1cb/users/6326fc96ca544c7283d384a13a13537a/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:14.586005 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fab30-7283-4bc4-8f43-b22b0cda5aff admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:14.596713 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795fab30-7283-4bc4-8f43-b22b0cda5aff admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:14.617861 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:14.657925 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb130f-a34e-4e5b-b207-2950f255b7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:14.662799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb130f-a34e-4e5b-b207-2950f255b7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:14.662799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb130f-a34e-4e5b-b207-2950f255b7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:14.662799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb130f-a34e-4e5b-b207-2950f255b7b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:14.662799 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb130f-a34e-4e5b-b207-2950f255b7b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:14.663547 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb130f-a34e-4e5b-b207-2950f255b7b2 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:14.675693 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1412/2838] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:18:14.680207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1935ee-f1c2-4d93-98af-e736c5865663 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:14.686859 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:14.694176 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1935ee-f1c2-4d93-98af-e736c5865663 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:14.699093 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7f3ae-be37-4a04-b6d6-e75b39cc18ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:14.700551 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7f3ae-be37-4a04-b6d6-e75b39cc18ab admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:14.700840 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7f3ae-be37-4a04-b6d6-e75b39cc18ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:14.701112 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7f3ae-be37-4a04-b6d6-e75b39cc18ab admin admin] PATH_INFO: `/v3/projects/76f0f040846a4c5a91a8c76a15e0e1cb/users/6326fc96ca544c7283d384a13a13537a/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:14.703136 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7f3ae-be37-4a04-b6d6-e75b39cc18ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=76f0f040846a4c5a91a8c76a15e0e1cb, user_id=6326fc96ca544c7283d384a13a13537a, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:14.705822 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7f3ae-be37-4a04-b6d6-e75b39cc18ab admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:14.716736 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1427/2839] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:18:14 2022] PUT /identity/v3/projects/76f0f040846a4c5a91a8c76a15e0e1cb/users/6326fc96ca544c7283d384a13a13537a/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:14.728441 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c94957-cade-4de8-b39a-a4be2f641f7a None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:14.734172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c94957-cade-4de8-b39a-a4be2f641f7a None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:14.734172 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c94957-cade-4de8-b39a-a4be2f641f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:14.758365 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-15c94957-cade-4de8-b39a-a4be2f641f7a None None] MFA Rules not processed for user `6326fc96ca544c7283d384a13a13537a`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:14.759767 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-15c94957-cade-4de8-b39a-a4be2f641f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:14.835241 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1413/2840] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:18:14.846260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7f3ae-be37-4a04-b6d6-e75b39cc18ab admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:14.849229 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-0ff7f3ae-be37-4a04-b6d6-e75b39cc18ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:14.909341 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7f3ae-be37-4a04-b6d6-e75b39cc18ab admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:14.913014 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9f3ed-40ff-4ced-bb23-346db4d08384 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:14.913625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9f3ed-40ff-4ced-bb23-346db4d08384 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:14.913905 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9f3ed-40ff-4ced-bb23-346db4d08384 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:14.914174 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9f3ed-40ff-4ced-bb23-346db4d08384 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:14.973849 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9f3ed-40ff-4ced-bb23-346db4d08384 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:14.974650 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9f3ed-40ff-4ced-bb23-346db4d08384 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:14.985045 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1428/2841] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:18:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:18:20.635056 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c94957-cade-4de8-b39a-a4be2f641f7a None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:20.636577 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-15c94957-cade-4de8-b39a-a4be2f641f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:20.649448 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c94957-cade-4de8-b39a-a4be2f641f7a None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:20.653052 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93d600-aa21-40ab-9668-a3736cb7d363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:20.653773 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93d600-aa21-40ab-9668-a3736cb7d363 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:20.654058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93d600-aa21-40ab-9668-a3736cb7d363 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:20.654338 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93d600-aa21-40ab-9668-a3736cb7d363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:20.663583 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93d600-aa21-40ab-9668-a3736cb7d363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:20.665001 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93d600-aa21-40ab-9668-a3736cb7d363 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:20.674823 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1414/2842] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:18:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:18:24.979928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:27.057585 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9f3ed-40ff-4ced-bb23-346db4d08384 service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:27.121540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9f3ed-40ff-4ced-bb23-346db4d08384 service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:27.126549 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531cd06-0b81-44f0-b8a5-e2fa7d560b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:27.127219 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531cd06-0b81-44f0-b8a5-e2fa7d560b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:27.127532 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531cd06-0b81-44f0-b8a5-e2fa7d560b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:27.127816 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531cd06-0b81-44f0-b8a5-e2fa7d560b22 admin admin] PATH_INFO: `/v3/users/bc32b6f30ce140509b7c3f48927eb1a5` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:27.129007 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531cd06-0b81-44f0-b8a5-e2fa7d560b22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc32b6f30ce140509b7c3f48927eb1a5)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:27.130952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531cd06-0b81-44f0-b8a5-e2fa7d560b22 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:27.376211 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d531cd06-0b81-44f0-b8a5-e2fa7d560b22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc32b6f30ce140509b7c3f48927eb1a5'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:27.383223 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d531cd06-0b81-44f0-b8a5-e2fa7d560b22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc32b6f30ce140509b7c3f48927eb1a5'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:27.500524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d531cd06-0b81-44f0-b8a5-e2fa7d560b22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc32b6f30ce140509b7c3f48927eb1a5'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:27.517273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d531cd06-0b81-44f0-b8a5-e2fa7d560b22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc32b6f30ce140509b7c3f48927eb1a5'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:27.521190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d531cd06-0b81-44f0-b8a5-e2fa7d560b22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc32b6f30ce140509b7c3f48927eb1a5'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:27.528236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1429/2843] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:18:27 2022] DELETE /identity/v3/users/bc32b6f30ce140509b7c3f48927eb1a5 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:27.533124 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1265c4f-32b6-42f0-bd60-0a479c0e88ce None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:27.533643 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1265c4f-32b6-42f0-bd60-0a479c0e88ce None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:27.533748 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1265c4f-32b6-42f0-bd60-0a479c0e88ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:27.557066 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-f1265c4f-32b6-42f0-bd60-0a479c0e88ce None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:27.558210 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f1265c4f-32b6-42f0-bd60-0a479c0e88ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:27.626307 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1415/2844] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:18:27.636438 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531cd06-0b81-44f0-b8a5-e2fa7d560b22 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:27.638677 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d531cd06-0b81-44f0-b8a5-e2fa7d560b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:27.698296 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531cd06-0b81-44f0-b8a5-e2fa7d560b22 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:27.702071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4dd4f-39f3-411c-b84b-75d09d7e3119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:27.702689 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4dd4f-39f3-411c-b84b-75d09d7e3119 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:27.702962 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4dd4f-39f3-411c-b84b-75d09d7e3119 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:27.703236 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4dd4f-39f3-411c-b84b-75d09d7e3119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:27.712523 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4dd4f-39f3-411c-b84b-75d09d7e3119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:27.714106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4dd4f-39f3-411c-b84b-75d09d7e3119 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:27.724506 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1430/2845] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:18:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:18:28.153722 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1265c4f-32b6-42f0-bd60-0a479c0e88ce None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:28.155258 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-f1265c4f-32b6-42f0-bd60-0a479c0e88ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:28.170866 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1265c4f-32b6-42f0-bd60-0a479c0e88ce None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:28.175691 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:28.176349 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:28.176631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:28.176907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] PATH_INFO: `/v3/projects/addb874bfc32490d97d0e15624ae8de8` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:28.179011 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=addb874bfc32490d97d0e15624ae8de8)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:28.181016 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:28.280669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:28.296937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'addb874bfc32490d97d0e15624ae8de8', 'request_id': 'req-242fd24b-397b-4501-8cf5-bbbc56930d1a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:28.303276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'addb874bfc32490d97d0e15624ae8de8', 'request_id': 'req-242fd24b-397b-4501-8cf5-bbbc56930d1a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:28.309500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'addb874bfc32490d97d0e15624ae8de8', 'request_id': 'req-242fd24b-397b-4501-8cf5-bbbc56930d1a'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:28.418115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] The token cache is being invalidate because project addb874bfc32490d97d0e15624ae8de8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:18:28.418508 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:28.419346 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:28.421028 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1416/2846] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:18:28 2022] DELETE /identity/v3/projects/addb874bfc32490d97d0e15624ae8de8 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:28.425191 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4dd4f-39f3-411c-b84b-75d09d7e3119 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:28.435355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4dd4f-39f3-411c-b84b-75d09d7e3119 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:28.456895 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:28.495507 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421b436-c837-46b4-b72c-c9512e4255cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:28.496135 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421b436-c837-46b4-b72c-c9512e4255cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:28.496420 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421b436-c837-46b4-b72c-c9512e4255cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:28.496846 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421b436-c837-46b4-b72c-c9512e4255cc admin admin] PATH_INFO: `/v3/users/6326fc96ca544c7283d384a13a13537a` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:28.497974 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421b436-c837-46b4-b72c-c9512e4255cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6326fc96ca544c7283d384a13a13537a)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:28.500103 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421b436-c837-46b4-b72c-c9512e4255cc admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:28.726971 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c421b436-c837-46b4-b72c-c9512e4255cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6326fc96ca544c7283d384a13a13537a'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:28.735242 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c421b436-c837-46b4-b72c-c9512e4255cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6326fc96ca544c7283d384a13a13537a'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:28.854694 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c421b436-c837-46b4-b72c-c9512e4255cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6326fc96ca544c7283d384a13a13537a'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:28.867889 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c421b436-c837-46b4-b72c-c9512e4255cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6326fc96ca544c7283d384a13a13537a'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:28.868540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-c421b436-c837-46b4-b72c-c9512e4255cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6326fc96ca544c7283d384a13a13537a'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:28.874540 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1431/2847] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:18:28 2022] DELETE /identity/v3/users/6326fc96ca544c7283d384a13a13537a => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:29.330646 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:29.333629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:29.346449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fd24b-397b-4501-8cf5-bbbc56930d1a admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:29.407794 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:29.408437 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:29.408719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:29.408983 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37 admin admin] PATH_INFO: `/v3/projects/76f0f040846a4c5a91a8c76a15e0e1cb` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:29.410136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76f0f040846a4c5a91a8c76a15e0e1cb)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:29.412064 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:29.522063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76f0f040846a4c5a91a8c76a15e0e1cb', 'request_id': 'req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:29.530968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76f0f040846a4c5a91a8c76a15e0e1cb', 'request_id': 'req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:29.536619 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76f0f040846a4c5a91a8c76a15e0e1cb', 'request_id': 'req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:29.592079 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37 admin admin] The token cache is being invalidate because project 76f0f040846a4c5a91a8c76a15e0e1cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:18:29.592299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:29.592895 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-988cb1fb-c0e5-4f9d-8bf8-1916e1dfee37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:29.594405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1417/2848] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:18:29 2022] DELETE /identity/v3/projects/76f0f040846a4c5a91a8c76a15e0e1cb => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:29.600481 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd9370-779b-4e84-9c2f-dc12fdb6d5ad None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:29.601011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd9370-779b-4e84-9c2f-dc12fdb6d5ad None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:29.601011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd9370-779b-4e84-9c2f-dc12fdb6d5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:29.623665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-09fd9370-779b-4e84-9c2f-dc12fdb6d5ad None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:29.624887 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-09fd9370-779b-4e84-9c2f-dc12fdb6d5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:29.686573 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1432/2849] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 15:18:29.695072 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ac3b0-4a2e-4fad-9347-fb588a019957 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:29.695380 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ac3b0-4a2e-4fad-9347-fb588a019957 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:29.695667 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ac3b0-4a2e-4fad-9347-fb588a019957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:29.719635 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-171ac3b0-4a2e-4fad-9347-fb588a019957 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:29.720855 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-171ac3b0-4a2e-4fad-9347-fb588a019957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:29.728758 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1418/2850] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:18:29.733921 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd9370-779b-4e84-9c2f-dc12fdb6d5ad None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:29.743817 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd9370-779b-4e84-9c2f-dc12fdb6d5ad None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:29.749037 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb081a-4e1d-4014-8a6a-8d39dcee0680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:29.749644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb081a-4e1d-4014-8a6a-8d39dcee0680 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:29.749927 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb081a-4e1d-4014-8a6a-8d39dcee0680 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:29.750204 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb081a-4e1d-4014-8a6a-8d39dcee0680 admin admin] PATH_INFO: `/v3/domains` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:29.750944 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb081a-4e1d-4014-8a6a-8d39dcee0680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92208) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:18:29.751252 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb081a-4e1d-4014-8a6a-8d39dcee0680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:29.752880 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb081a-4e1d-4014-8a6a-8d39dcee0680 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:29.765314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1433/2851] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:29.769751 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ac3b0-4a2e-4fad-9347-fb588a019957 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:29.779104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ac3b0-4a2e-4fad-9347-fb588a019957 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:29.783806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa2e77-eb9a-428c-af50-6fda44c941fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:29.784401 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa2e77-eb9a-428c-af50-6fda44c941fb admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:29.784682 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa2e77-eb9a-428c-af50-6fda44c941fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:29.784953 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa2e77-eb9a-428c-af50-6fda44c941fb admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:29.785614 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa2e77-eb9a-428c-af50-6fda44c941fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:29.787507 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa2e77-eb9a-428c-af50-6fda44c941fb admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:29.891669 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1419/2852] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:18:29 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:18:29.895946 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb081a-4e1d-4014-8a6a-8d39dcee0680 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:29.906346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb081a-4e1d-4014-8a6a-8d39dcee0680 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:29.932797 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:29.963897 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa30df9-51a6-48b4-88f6-f8ac0a48679a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:29.964509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa30df9-51a6-48b4-88f6-f8ac0a48679a admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:29.964791 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa30df9-51a6-48b4-88f6-f8ac0a48679a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:29.965066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa30df9-51a6-48b4-88f6-f8ac0a48679a admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:29.965868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa30df9-51a6-48b4-88f6-f8ac0a48679a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:29.967810 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa30df9-51a6-48b4-88f6-f8ac0a48679a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:30.066512 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1434/2853] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:18:29 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:18:30.070749 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa2e77-eb9a-428c-af50-6fda44c941fb admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:30.085789 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa2e77-eb9a-428c-af50-6fda44c941fb admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:30.087227 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282b3e3-6937-46d9-a1d9-e213768828bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:30.087870 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282b3e3-6937-46d9-a1d9-e213768828bb admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:30.088158 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282b3e3-6937-46d9-a1d9-e213768828bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:30.088466 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282b3e3-6937-46d9-a1d9-e213768828bb admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:30.089170 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282b3e3-6937-46d9-a1d9-e213768828bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:30.090947 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282b3e3-6937-46d9-a1d9-e213768828bb admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:30.101465 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1420/2854] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:18:30.106474 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa30df9-51a6-48b4-88f6-f8ac0a48679a admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:30.116755 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa30df9-51a6-48b4-88f6-f8ac0a48679a admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:30.122044 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f628c5d-4b92-4f1f-abc6-f1b5fc3aa1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:30.126381 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f628c5d-4b92-4f1f-abc6-f1b5fc3aa1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:30.126663 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f628c5d-4b92-4f1f-abc6-f1b5fc3aa1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:30.126963 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f628c5d-4b92-4f1f-abc6-f1b5fc3aa1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:30.127764 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f628c5d-4b92-4f1f-abc6-f1b5fc3aa1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:30.129434 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f628c5d-4b92-4f1f-abc6-f1b5fc3aa1d1 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:30.140285 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1435/2855] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:18:30.144688 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282b3e3-6937-46d9-a1d9-e213768828bb admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:30.155545 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282b3e3-6937-46d9-a1d9-e213768828bb admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:30.160136 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db473193-4482-46c1-b707-fee9e1e17eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:30.160797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db473193-4482-46c1-b707-fee9e1e17eda admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:30.161076 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db473193-4482-46c1-b707-fee9e1e17eda admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:30.161348 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db473193-4482-46c1-b707-fee9e1e17eda admin admin] PATH_INFO: `/v3/projects/07106c49e4be4c50a6cfd1342dc8ea6c/users/5d81a0d1d78f45b095d6a955b8d460ce/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:30.163743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:30.177123 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db473193-4482-46c1-b707-fee9e1e17eda admin admin] RBAC: Authorizing `identity:create_grant(project_id=07106c49e4be4c50a6cfd1342dc8ea6c, user_id=5d81a0d1d78f45b095d6a955b8d460ce, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:30.179803 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db473193-4482-46c1-b707-fee9e1e17eda admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:30.289108 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1421/2856] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:18:30 2022] PUT /identity/v3/projects/07106c49e4be4c50a6cfd1342dc8ea6c/users/5d81a0d1d78f45b095d6a955b8d460ce/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:18:30.302373 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409900db-12d2-4ad7-9562-da2c66a235a5 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:30.303317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409900db-12d2-4ad7-9562-da2c66a235a5 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:30.303317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409900db-12d2-4ad7-9562-da2c66a235a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:30.332144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-409900db-12d2-4ad7-9562-da2c66a235a5 None None] MFA Rules not processed for user `5d81a0d1d78f45b095d6a955b8d460ce`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:30.333524 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-409900db-12d2-4ad7-9562-da2c66a235a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:30.400644 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1436/2857] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:18:30.405616 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db473193-4482-46c1-b707-fee9e1e17eda admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:30.467174 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db473193-4482-46c1-b707-fee9e1e17eda admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:30.471903 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d472129-c772-4e65-8320-5a26f7915ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:30.472515 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d472129-c772-4e65-8320-5a26f7915ddb admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:30.472798 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d472129-c772-4e65-8320-5a26f7915ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:30.473075 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d472129-c772-4e65-8320-5a26f7915ddb admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:30.473741 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d472129-c772-4e65-8320-5a26f7915ddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:30.475660 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d472129-c772-4e65-8320-5a26f7915ddb admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:30.553468 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1422/2858] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:18:30 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:18:30.557885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409900db-12d2-4ad7-9562-da2c66a235a5 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:30.570921 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409900db-12d2-4ad7-9562-da2c66a235a5 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:30.628731 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe788dd9-8789-421a-91c5-b63edd6edfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:30.629467 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe788dd9-8789-421a-91c5-b63edd6edfff admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:30.629602 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe788dd9-8789-421a-91c5-b63edd6edfff admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:30.629872 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe788dd9-8789-421a-91c5-b63edd6edfff admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:30.630533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe788dd9-8789-421a-91c5-b63edd6edfff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:30.632471 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe788dd9-8789-421a-91c5-b63edd6edfff admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:30.770563 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1437/2859] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:18:30 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:18:30.774885 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d472129-c772-4e65-8320-5a26f7915ddb admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:30.786491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d472129-c772-4e65-8320-5a26f7915ddb admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:30.791095 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961fb80-4fd3-4fea-b2c4-0e8ee65954e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:30.791751 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961fb80-4fd3-4fea-b2c4-0e8ee65954e4 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:30.792058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961fb80-4fd3-4fea-b2c4-0e8ee65954e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:30.792394 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961fb80-4fd3-4fea-b2c4-0e8ee65954e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:30.793224 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961fb80-4fd3-4fea-b2c4-0e8ee65954e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:30.795676 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961fb80-4fd3-4fea-b2c4-0e8ee65954e4 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:30.806797 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1423/2860] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:18:30.811583 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe788dd9-8789-421a-91c5-b63edd6edfff admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:30.821692 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe788dd9-8789-421a-91c5-b63edd6edfff admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:30.827935 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fc1d0-1c56-462a-aba8-7f1263dbbeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:30.828720 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fc1d0-1c56-462a-aba8-7f1263dbbeeb admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:30.828998 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fc1d0-1c56-462a-aba8-7f1263dbbeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:30.829292 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fc1d0-1c56-462a-aba8-7f1263dbbeeb admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:30.830028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fc1d0-1c56-462a-aba8-7f1263dbbeeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:30.832349 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fc1d0-1c56-462a-aba8-7f1263dbbeeb admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:30.843427 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1438/2861] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:18:30.847846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961fb80-4fd3-4fea-b2c4-0e8ee65954e4 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:30.858824 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961fb80-4fd3-4fea-b2c4-0e8ee65954e4 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:30.863391 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3871a-51c4-4ae2-8f40-8ebead60f7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:30.864098 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3871a-51c4-4ae2-8f40-8ebead60f7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:30.864384 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3871a-51c4-4ae2-8f40-8ebead60f7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:30.864668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3871a-51c4-4ae2-8f40-8ebead60f7f7 admin admin] PATH_INFO: `/v3/projects/d801a4657e2540c5944c2925e7167aac/users/77c88cf7e3de48bfa580a82a2e2b8efc/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:30.880362 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3871a-51c4-4ae2-8f40-8ebead60f7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d801a4657e2540c5944c2925e7167aac, user_id=77c88cf7e3de48bfa580a82a2e2b8efc, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:30.883057 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3871a-51c4-4ae2-8f40-8ebead60f7f7 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:30.974664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1424/2862] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:18:30 2022] PUT /identity/v3/projects/d801a4657e2540c5944c2925e7167aac/users/77c88cf7e3de48bfa580a82a2e2b8efc/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:30.986184 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a262ce1-d618-468f-aead-5172a155a8df None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:30.986476 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a262ce1-d618-468f-aead-5172a155a8df None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:30.986839 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a262ce1-d618-468f-aead-5172a155a8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:31.016053 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-1a262ce1-d618-468f-aead-5172a155a8df None None] MFA Rules not processed for user `77c88cf7e3de48bfa580a82a2e2b8efc`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:31.017341 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-1a262ce1-d618-468f-aead-5172a155a8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:31.021571 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:31.084237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1439/2863] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:18:31.088818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3871a-51c4-4ae2-8f40-8ebead60f7f7 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:31.153242 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3871a-51c4-4ae2-8f40-8ebead60f7f7 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:31.157890 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fed156-0168-420c-9ed6-784c35e6f35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:31.158507 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fed156-0168-420c-9ed6-784c35e6f35b admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:31.158786 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fed156-0168-420c-9ed6-784c35e6f35b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:31.159059 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fed156-0168-420c-9ed6-784c35e6f35b admin admin] PATH_INFO: `/v3/projects` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:31.159784 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fed156-0168-420c-9ed6-784c35e6f35b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:31.161907 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fed156-0168-420c-9ed6-784c35e6f35b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:31.268090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1425/2864] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:18:31 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:18:31.272339 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a262ce1-d618-468f-aead-5172a155a8df None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:31.284592 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a262ce1-d618-468f-aead-5172a155a8df None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:31.344164 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33db43-b8cd-43f6-b966-6e402730e5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:31.344782 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33db43-b8cd-43f6-b966-6e402730e5f0 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:31.345066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33db43-b8cd-43f6-b966-6e402730e5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:31.345359 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33db43-b8cd-43f6-b966-6e402730e5f0 admin admin] PATH_INFO: `/v3/users` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:31.346027 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33db43-b8cd-43f6-b966-6e402730e5f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:31.347952 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33db43-b8cd-43f6-b966-6e402730e5f0 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:31.460993 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1440/2865] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:18:31 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 15:18:31.465752 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fed156-0168-420c-9ed6-784c35e6f35b admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:31.476599 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fed156-0168-420c-9ed6-784c35e6f35b admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:31.481934 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bd5b8-5b4e-4eea-b98d-38f193f4e314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:31.482550 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd5b8-5b4e-4eea-b98d-38f193f4e314 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:31.482837 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd5b8-5b4e-4eea-b98d-38f193f4e314 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:31.483119 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd5b8-5b4e-4eea-b98d-38f193f4e314 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:31.483861 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bd5b8-5b4e-4eea-b98d-38f193f4e314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:31.485452 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bd5b8-5b4e-4eea-b98d-38f193f4e314 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:31.496232 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1426/2866] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:18:31.500604 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33db43-b8cd-43f6-b966-6e402730e5f0 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:31.510909 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33db43-b8cd-43f6-b966-6e402730e5f0 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:31.516314 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67ae50-b6e8-40f9-80bb-3a2bf4655cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:31.516991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67ae50-b6e8-40f9-80bb-3a2bf4655cb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:31.517265 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67ae50-b6e8-40f9-80bb-3a2bf4655cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:31.517544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67ae50-b6e8-40f9-80bb-3a2bf4655cb8 admin admin] PATH_INFO: `/v3/projects/f4e1fb0103a74173b256d59ee0dc981e/users/132b3ff923174904a7d23ed9f6e02913/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:31.533729 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67ae50-b6e8-40f9-80bb-3a2bf4655cb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e1fb0103a74173b256d59ee0dc981e, user_id=132b3ff923174904a7d23ed9f6e02913, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:31.536346 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67ae50-b6e8-40f9-80bb-3a2bf4655cb8 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:31.615051 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1441/2867] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:18:31 2022] PUT /identity/v3/projects/f4e1fb0103a74173b256d59ee0dc981e/users/132b3ff923174904a7d23ed9f6e02913/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:18:31.619051 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bd5b8-5b4e-4eea-b98d-38f193f4e314 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:31.629140 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bd5b8-5b4e-4eea-b98d-38f193f4e314 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:31.632860 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:31.685395 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab90742-849c-4de7-8db7-afd5efd1ca21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:31.686014 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab90742-849c-4de7-8db7-afd5efd1ca21 admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:31.686298 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab90742-849c-4de7-8db7-afd5efd1ca21 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:31.686576 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab90742-849c-4de7-8db7-afd5efd1ca21 admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:31.687272 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab90742-849c-4de7-8db7-afd5efd1ca21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:31.689004 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab90742-849c-4de7-8db7-afd5efd1ca21 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:31.699034 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1427/2868] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:18:31.704330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67ae50-b6e8-40f9-80bb-3a2bf4655cb8 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:31.718686 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67ae50-b6e8-40f9-80bb-3a2bf4655cb8 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:31.723543 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcd03a-17b2-43cf-b1f9-cce4bdb29eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:31.728365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcd03a-17b2-43cf-b1f9-cce4bdb29eba admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:31.728365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcd03a-17b2-43cf-b1f9-cce4bdb29eba admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:31.728365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcd03a-17b2-43cf-b1f9-cce4bdb29eba admin admin] PATH_INFO: `/v3/projects/f4e1fb0103a74173b256d59ee0dc981e/users/132b3ff923174904a7d23ed9f6e02913/roles/2bb293e71ec0464fb7f6fa454e6b3f51` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:31.728365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bcd03a-17b2-43cf-b1f9-cce4bdb29eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e1fb0103a74173b256d59ee0dc981e, user_id=132b3ff923174904a7d23ed9f6e02913, role_id=2bb293e71ec0464fb7f6fa454e6b3f51)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:31.729748 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bcd03a-17b2-43cf-b1f9-cce4bdb29eba admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:31.740560 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1442/2869] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:18:31 2022] PUT /identity/v3/projects/f4e1fb0103a74173b256d59ee0dc981e/users/132b3ff923174904a7d23ed9f6e02913/roles/2bb293e71ec0464fb7f6fa454e6b3f51 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:18:31.751672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e341b6c-2812-4ed7-8c00-e479a6dcb445 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:31.752408 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e341b6c-2812-4ed7-8c00-e479a6dcb445 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:31.752764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e341b6c-2812-4ed7-8c00-e479a6dcb445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:31.782881 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-4e341b6c-2812-4ed7-8c00-e479a6dcb445 None None] MFA Rules not processed for user `132b3ff923174904a7d23ed9f6e02913`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:31.784201 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4e341b6c-2812-4ed7-8c00-e479a6dcb445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:31.852854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1428/2870] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:18:31.865353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcd03a-17b2-43cf-b1f9-cce4bdb29eba admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:31.867096 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-09bcd03a-17b2-43cf-b1f9-cce4bdb29eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:31.928711 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcd03a-17b2-43cf-b1f9-cce4bdb29eba admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:31.932578 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88f297-2e30-4629-891a-9e3d2e72349e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:31.933221 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88f297-2e30-4629-891a-9e3d2e72349e service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:31.933502 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88f297-2e30-4629-891a-9e3d2e72349e service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:31.933811 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88f297-2e30-4629-891a-9e3d2e72349e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:31.944457 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:31.996161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88f297-2e30-4629-891a-9e3d2e72349e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:31.998509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88f297-2e30-4629-891a-9e3d2e72349e service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:32.007550 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1443/2871] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:18:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:18:34.466500 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e341b6c-2812-4ed7-8c00-e479a6dcb445 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:34.467968 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-4e341b6c-2812-4ed7-8c00-e479a6dcb445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:34.480241 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e341b6c-2812-4ed7-8c00-e479a6dcb445 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:34.483854 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14bbd1c-d57f-4ded-95de-dff6710f1857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:34.484491 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bbd1c-d57f-4ded-95de-dff6710f1857 service cinder] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:34.484773 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bbd1c-d57f-4ded-95de-dff6710f1857 service cinder] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:34.485050 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bbd1c-d57f-4ded-95de-dff6710f1857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:34.543471 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14bbd1c-d57f-4ded-95de-dff6710f1857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:34.544942 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14bbd1c-d57f-4ded-95de-dff6710f1857 service cinder] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:34.554624 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1429/2872] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:18:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:18:36.278894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88f297-2e30-4629-891a-9e3d2e72349e service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:36.278894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88f297-2e30-4629-891a-9e3d2e72349e service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:36.278894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a802c49-eb64-40a7-a81c-e85a46941e97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:36.278894 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a802c49-eb64-40a7-a81c-e85a46941e97 service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:36.279816 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a802c49-eb64-40a7-a81c-e85a46941e97 service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:36.279816 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a802c49-eb64-40a7-a81c-e85a46941e97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:36.289335 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a802c49-eb64-40a7-a81c-e85a46941e97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:36.290923 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a802c49-eb64-40a7-a81c-e85a46941e97 service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:36.301992 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1444/2873] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:18:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:18:41.540178 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14bbd1c-d57f-4ded-95de-dff6710f1857 service cinder] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:41.570898 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=100 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:41.604610 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14bbd1c-d57f-4ded-95de-dff6710f1857 service cinder] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:41.609165 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b574fa8-35d9-45e8-9a29-1eccc0bbdf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:41.609808 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b574fa8-35d9-45e8-9a29-1eccc0bbdf19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:41.610100 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b574fa8-35d9-45e8-9a29-1eccc0bbdf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:41.611115 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b574fa8-35d9-45e8-9a29-1eccc0bbdf19 admin admin] PATH_INFO: `/v3/users/5d81a0d1d78f45b095d6a955b8d460ce` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:41.612357 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b574fa8-35d9-45e8-9a29-1eccc0bbdf19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d81a0d1d78f45b095d6a955b8d460ce)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:41.614315 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b574fa8-35d9-45e8-9a29-1eccc0bbdf19 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:41.844969 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3b574fa8-35d9-45e8-9a29-1eccc0bbdf19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d81a0d1d78f45b095d6a955b8d460ce'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:41.845640 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3b574fa8-35d9-45e8-9a29-1eccc0bbdf19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d81a0d1d78f45b095d6a955b8d460ce'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:41.846276 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3b574fa8-35d9-45e8-9a29-1eccc0bbdf19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d81a0d1d78f45b095d6a955b8d460ce'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:41.853304 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3b574fa8-35d9-45e8-9a29-1eccc0bbdf19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d81a0d1d78f45b095d6a955b8d460ce'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:41.866078 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-3b574fa8-35d9-45e8-9a29-1eccc0bbdf19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d81a0d1d78f45b095d6a955b8d460ce'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:41.934229 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1430/2874] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:18:41 2022] DELETE /identity/v3/users/5d81a0d1d78f45b095d6a955b8d460ce => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:41.943023 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c1767-f089-4aa2-ab48-5c9f81e34b8f None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:41.943589 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c1767-f089-4aa2-ab48-5c9f81e34b8f None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:41.943716 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c1767-f089-4aa2-ab48-5c9f81e34b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:41.963022 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-b81c1767-f089-4aa2-ab48-5c9f81e34b8f None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:41.964260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b81c1767-f089-4aa2-ab48-5c9f81e34b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:42.023280 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1445/2875] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:18:42.033191 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b574fa8-35d9-45e8-9a29-1eccc0bbdf19 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:42.034611 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3b574fa8-35d9-45e8-9a29-1eccc0bbdf19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:42.093637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b574fa8-35d9-45e8-9a29-1eccc0bbdf19 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:42.097382 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc5889-1e6b-42d7-81f0-9c201aadd9bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:42.097938 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc5889-1e6b-42d7-81f0-9c201aadd9bc service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:42.098221 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc5889-1e6b-42d7-81f0-9c201aadd9bc service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:42.098497 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc5889-1e6b-42d7-81f0-9c201aadd9bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:42.108058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc5889-1e6b-42d7-81f0-9c201aadd9bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:42.109482 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc5889-1e6b-42d7-81f0-9c201aadd9bc service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:42.119612 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1431/2876] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:18:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:18:42.566817 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c1767-f089-4aa2-ab48-5c9f81e34b8f None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:42.568347 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-b81c1767-f089-4aa2-ab48-5c9f81e34b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:42.582067 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c1767-f089-4aa2-ab48-5c9f81e34b8f None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:42.586945 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:42.587778 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:42.591634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:42.591634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] PATH_INFO: `/v3/projects/07106c49e4be4c50a6cfd1342dc8ea6c` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:42.591634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07106c49e4be4c50a6cfd1342dc8ea6c)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:42.592330 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:42.727866 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07106c49e4be4c50a6cfd1342dc8ea6c', 'request_id': 'req-e465492c-0649-44d5-89a4-e04c2d19e5e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:42.733625 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07106c49e4be4c50a6cfd1342dc8ea6c', 'request_id': 'req-e465492c-0649-44d5-89a4-e04c2d19e5e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:42.820757 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07106c49e4be4c50a6cfd1342dc8ea6c', 'request_id': 'req-e465492c-0649-44d5-89a4-e04c2d19e5e1'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:42.838537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] The token cache is being invalidate because project 07106c49e4be4c50a6cfd1342dc8ea6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:18:42.838824 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:42.839509 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:42.841061 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1446/2877] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:18:42 2022] DELETE /identity/v3/projects/07106c49e4be4c50a6cfd1342dc8ea6c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:18:42.845096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc5889-1e6b-42d7-81f0-9c201aadd9bc service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:42.855503 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc5889-1e6b-42d7-81f0-9c201aadd9bc service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:42.902449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:42.916066 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ba55b-0e6a-467d-ae91-05dc192521f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:42.916706 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ba55b-0e6a-467d-ae91-05dc192521f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:42.917030 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ba55b-0e6a-467d-ae91-05dc192521f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:42.917268 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ba55b-0e6a-467d-ae91-05dc192521f3 admin admin] PATH_INFO: `/v3/users/77c88cf7e3de48bfa580a82a2e2b8efc` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:42.918499 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ba55b-0e6a-467d-ae91-05dc192521f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77c88cf7e3de48bfa580a82a2e2b8efc)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:42.920555 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ba55b-0e6a-467d-ae91-05dc192521f3 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:43.103073 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a46ba55b-0e6a-467d-ae91-05dc192521f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77c88cf7e3de48bfa580a82a2e2b8efc'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:43.104012 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a46ba55b-0e6a-467d-ae91-05dc192521f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77c88cf7e3de48bfa580a82a2e2b8efc'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:43.104736 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a46ba55b-0e6a-467d-ae91-05dc192521f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77c88cf7e3de48bfa580a82a2e2b8efc'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:43.113554 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a46ba55b-0e6a-467d-ae91-05dc192521f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77c88cf7e3de48bfa580a82a2e2b8efc'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:43.129214 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-a46ba55b-0e6a-467d-ae91-05dc192521f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77c88cf7e3de48bfa580a82a2e2b8efc'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:43.203285 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1432/2878] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:18:42 2022] DELETE /identity/v3/users/77c88cf7e3de48bfa580a82a2e2b8efc => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:43.913536 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:43.915011 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:43.929601 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465492c-0649-44d5-89a4-e04c2d19e5e1 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:43.950183 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:43.990634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:43.991274 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:43.991591 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:43.991868 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] PATH_INFO: `/v3/projects/d801a4657e2540c5944c2925e7167aac` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:43.993059 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d801a4657e2540c5944c2925e7167aac)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:43.995008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:44.130879 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd801a4657e2540c5944c2925e7167aac', 'request_id': 'req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:44.136401 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd801a4657e2540c5944c2925e7167aac', 'request_id': 'req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:44.214106 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd801a4657e2540c5944c2925e7167aac', 'request_id': 'req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:44.226105 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] The token cache is being invalidate because project d801a4657e2540c5944c2925e7167aac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:18:44.226406 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:44.227030 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:44.229050 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1447/2879] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:18:43 2022] DELETE /identity/v3/projects/d801a4657e2540c5944c2925e7167aac => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:18:44.232973 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ba55b-0e6a-467d-ae91-05dc192521f3 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:44.239185 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-a46ba55b-0e6a-467d-ae91-05dc192521f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:44.255637 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ba55b-0e6a-467d-ae91-05dc192521f3 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:44.316740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef110c-a9dc-4042-8506-5e1d6b3e2680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:44.317392 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef110c-a9dc-4042-8506-5e1d6b3e2680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:44.317675 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef110c-a9dc-4042-8506-5e1d6b3e2680 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:44.317952 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef110c-a9dc-4042-8506-5e1d6b3e2680 admin admin] PATH_INFO: `/v3/users/132b3ff923174904a7d23ed9f6e02913` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:44.319185 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef110c-a9dc-4042-8506-5e1d6b3e2680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=132b3ff923174904a7d23ed9f6e02913)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:44.321167 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef110c-a9dc-4042-8506-5e1d6b3e2680 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:44.529972 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-96ef110c-a9dc-4042-8506-5e1d6b3e2680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '132b3ff923174904a7d23ed9f6e02913'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:44.530629 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-96ef110c-a9dc-4042-8506-5e1d6b3e2680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '132b3ff923174904a7d23ed9f6e02913'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:44.531337 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-96ef110c-a9dc-4042-8506-5e1d6b3e2680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '132b3ff923174904a7d23ed9f6e02913'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:44.538396 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-96ef110c-a9dc-4042-8506-5e1d6b3e2680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '132b3ff923174904a7d23ed9f6e02913'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:44.548773 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:44.551103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-96ef110c-a9dc-4042-8506-5e1d6b3e2680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '132b3ff923174904a7d23ed9f6e02913'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:44.641030 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1433/2880] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:18:44 2022] DELETE /identity/v3/users/132b3ff923174904a7d23ed9f6e02913 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 15:18:45.126407 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:45.128008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:45.140928 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2495e1f-bb13-4d8c-b2eb-024eaa7448c7 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:45.199486 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:45.200155 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:45.200434 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:45.200707 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a admin admin] PATH_INFO: `/v3/projects/f4e1fb0103a74173b256d59ee0dc981e` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:45.202065 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e1fb0103a74173b256d59ee0dc981e)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:45.204069 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:45.335870 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e1fb0103a74173b256d59ee0dc981e', 'request_id': 'req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:45.339938 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:45.341836 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e1fb0103a74173b256d59ee0dc981e', 'request_id': 'req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:45.442688 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e1fb0103a74173b256d59ee0dc981e', 'request_id': 'req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:45.457474 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a admin admin] The token cache is being invalidate because project f4e1fb0103a74173b256d59ee0dc981e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92208) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:18:45.458617 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:45.458617 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-d6c12a7c-3e6b-4d78-9ec4-4ae08565151a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:18:45.460121 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1448/2881] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:18:45 2022] DELETE /identity/v3/projects/f4e1fb0103a74173b256d59ee0dc981e => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:45.464928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c042a45-6b8b-4647-ab73-6d4272a33fbb None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:45.465429 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c042a45-6b8b-4647-ab73-6d4272a33fbb None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:45.465540 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c042a45-6b8b-4647-ab73-6d4272a33fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:45.486218 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-8c042a45-6b8b-4647-ab73-6d4272a33fbb None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:45.488299 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-8c042a45-6b8b-4647-ab73-6d4272a33fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:45.549893 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1434/2882] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:18:45.558127 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b20d3e-f7bd-464b-a848-d2c2bea45016 None None] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:45.562028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b20d3e-f7bd-464b-a848-d2c2bea45016 None None] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:45.562028 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b20d3e-f7bd-464b-a848-d2c2bea45016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:45.578315 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.auth.core [None req-99b20d3e-f7bd-464b-a848-d2c2bea45016 None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92208) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:45.579508 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-99b20d3e-f7bd-464b-a848-d2c2bea45016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:45.588209 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1449/2883] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:18:45.592668 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c042a45-6b8b-4647-ab73-6d4272a33fbb None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:45.603305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c042a45-6b8b-4647-ab73-6d4272a33fbb None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:45.608203 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c2fd7-19d6-490c-8578-cb5d79c888fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:45.608818 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c2fd7-19d6-490c-8578-cb5d79c888fd admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:45.609103 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c2fd7-19d6-490c-8578-cb5d79c888fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:45.609381 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c2fd7-19d6-490c-8578-cb5d79c888fd admin admin] PATH_INFO: `/v3/domains` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:45.610093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c2fd7-19d6-490c-8578-cb5d79c888fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92209) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 15:18:45.610390 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c2fd7-19d6-490c-8578-cb5d79c888fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:45.612144 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c2fd7-19d6-490c-8578-cb5d79c888fd admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:45.623723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1435/2884] 10.222.0.94 () {58 vars in 1153 bytes} [Tue Jul 26 15:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 15:18:45.634291 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b20d3e-f7bd-464b-a848-d2c2bea45016 None None] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:45.644038 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b20d3e-f7bd-464b-a848-d2c2bea45016 None None] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:45.652516 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41477b-e517-48c4-a018-6d4534434384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:45.653153 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41477b-e517-48c4-a018-6d4534434384 admin admin] REQUEST_METHOD: `POST` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:45.653432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41477b-e517-48c4-a018-6d4534434384 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:45.653701 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41477b-e517-48c4-a018-6d4534434384 admin admin] PATH_INFO: `/v3/projects` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:45.654368 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41477b-e517-48c4-a018-6d4534434384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:45.656351 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41477b-e517-48c4-a018-6d4534434384 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:45.759370 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1450/2885] 10.222.0.94 () {60 vars in 1153 bytes} [Tue Jul 26 15:18:45 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:18:45.763593 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c2fd7-19d6-490c-8578-cb5d79c888fd admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:45.774454 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c2fd7-19d6-490c-8578-cb5d79c888fd admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:45.836963 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a9a53-cfb1-40b4-a370-ce8795734d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:45.837585 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9a53-cfb1-40b4-a370-ce8795734d3b admin admin] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:45.837873 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9a53-cfb1-40b4-a370-ce8795734d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:45.838152 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9a53-cfb1-40b4-a370-ce8795734d3b admin admin] PATH_INFO: `/v3/users` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:45.838826 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a9a53-cfb1-40b4-a370-ce8795734d3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:45.840750 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a9a53-cfb1-40b4-a370-ce8795734d3b admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:45.957634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1436/2886] 10.222.0.94 () {60 vars in 1144 bytes} [Tue Jul 26 15:18:45 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 15:18:45.961987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41477b-e517-48c4-a018-6d4534434384 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:45.973429 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41477b-e517-48c4-a018-6d4534434384 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:45.978129 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982dccac-ae0e-4819-8822-10ab4286d265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:45.979017 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982dccac-ae0e-4819-8822-10ab4286d265 admin admin] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:45.979298 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982dccac-ae0e-4819-8822-10ab4286d265 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:45.979612 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982dccac-ae0e-4819-8822-10ab4286d265 admin admin] PATH_INFO: `/v3/roles` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:45.980305 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982dccac-ae0e-4819-8822-10ab4286d265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:45.981898 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982dccac-ae0e-4819-8822-10ab4286d265 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:45.993569 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1451/2887] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:18:45.997657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a9a53-cfb1-40b4-a370-ce8795734d3b admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:46.008349 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a9a53-cfb1-40b4-a370-ce8795734d3b admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:46.013013 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03ec1b-9241-481e-a2be-08371b58423c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:46.013631 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03ec1b-9241-481e-a2be-08371b58423c admin admin] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:46.013911 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03ec1b-9241-481e-a2be-08371b58423c admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:46.014239 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03ec1b-9241-481e-a2be-08371b58423c admin admin] PATH_INFO: `/v3/roles` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:46.014886 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03ec1b-9241-481e-a2be-08371b58423c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:46.016503 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03ec1b-9241-481e-a2be-08371b58423c admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:46.027269 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1437/2888] 10.222.0.94 () {58 vars in 1122 bytes} [Tue Jul 26 15:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 15:18:46.031943 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982dccac-ae0e-4819-8822-10ab4286d265 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:46.042811 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982dccac-ae0e-4819-8822-10ab4286d265 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:46.047885 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae477b6-4f08-4e98-96ca-0e21d9ab36a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:46.048544 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae477b6-4f08-4e98-96ca-0e21d9ab36a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:46.048966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae477b6-4f08-4e98-96ca-0e21d9ab36a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:46.049760 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae477b6-4f08-4e98-96ca-0e21d9ab36a4 admin admin] PATH_INFO: `/v3/projects/8c2393ea2f0d4b44bc10198887c931b9/users/a2a1491c324f4306af9f2c96587213cc/roles/6a924f1d619643b3be92bab6fddb3eb2` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:46.067317 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae477b6-4f08-4e98-96ca-0e21d9ab36a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2393ea2f0d4b44bc10198887c931b9, user_id=a2a1491c324f4306af9f2c96587213cc, role_id=6a924f1d619643b3be92bab6fddb3eb2)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:46.070034 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae477b6-4f08-4e98-96ca-0e21d9ab36a4 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:46.248965 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1452/2889] 10.222.0.94 () {60 vars in 1483 bytes} [Tue Jul 26 15:18:46 2022] PUT /identity/v3/projects/8c2393ea2f0d4b44bc10198887c931b9/users/a2a1491c324f4306af9f2c96587213cc/roles/6a924f1d619643b3be92bab6fddb3eb2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:18:46.260156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edaffd-1ebd-4edf-8ade-30dd6d66ee74 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:46.260447 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edaffd-1ebd-4edf-8ade-30dd6d66ee74 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:46.260723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edaffd-1ebd-4edf-8ade-30dd6d66ee74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:46.289598 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-49edaffd-1ebd-4edf-8ade-30dd6d66ee74 None None] MFA Rules not processed for user `a2a1491c324f4306af9f2c96587213cc`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:18:46.290816 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-49edaffd-1ebd-4edf-8ade-30dd6d66ee74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:46.335842 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:46.358846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1438/2890] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:18:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:18:46.371090 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae477b6-4f08-4e98-96ca-0e21d9ab36a4 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:18:46.379273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-cae477b6-4f08-4e98-96ca-0e21d9ab36a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:18:46.432533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae477b6-4f08-4e98-96ca-0e21d9ab36a4 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:18:46.436671 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fc7b8-05c0-4fac-a8fc-7efbf3bcec7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7448959048f746a0ae1588ff839deb0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:18:46.436671 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fc7b8-05c0-4fac-a8fc-7efbf3bcec7e service cinder] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:18:46.437066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fc7b8-05c0-4fac-a8fc-7efbf3bcec7e service cinder] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:18:46.437173 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fc7b8-05c0-4fac-a8fc-7efbf3bcec7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:18:46.446634 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fc7b8-05c0-4fac-a8fc-7efbf3bcec7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:18:46.448201 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fc7b8-05c0-4fac-a8fc-7efbf3bcec7e service cinder] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:18:46.457893 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1453/2891] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:18:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:18:56.453102 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:DELETE=2,keystone:SELECT=58,keystone:INSERT=3 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:18:56.688092 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=8 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:09.404480 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25345a7-501e-4de1-8a5e-0fd2cc7edad8 None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:09.405134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25345a7-501e-4de1-8a5e-0fd2cc7edad8 None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:09.405134 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25345a7-501e-4de1-8a5e-0fd2cc7edad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:09.426781 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-b25345a7-501e-4de1-8a5e-0fd2cc7edad8 None None] MFA Rules not processed for user `c11f43d5614d46cd996789819075520d`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:19:09.428291 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b25345a7-501e-4de1-8a5e-0fd2cc7edad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:09.487524 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1439/2892] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:19:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:19:09.821592 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fc7b8-05c0-4fac-a8fc-7efbf3bcec7e service cinder] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:09.823799 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d58fc7b8-05c0-4fac-a8fc-7efbf3bcec7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:09.884179 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fc7b8-05c0-4fac-a8fc-7efbf3bcec7e service cinder] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:09.888228 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294cb48-ada4-4764-b4c8-106f75b010b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:09.888881 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294cb48-ada4-4764-b4c8-106f75b010b5 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:09.889161 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294cb48-ada4-4764-b4c8-106f75b010b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:09.889436 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294cb48-ada4-4764-b4c8-106f75b010b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:09.891260 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-c294cb48-ada4-4764-b4c8-106f75b010b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:09.901927 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294cb48-ada4-4764-b4c8-106f75b010b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:09.903378 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294cb48-ada4-4764-b4c8-106f75b010b5 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:09.911767 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1454/2893] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:19:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:19:19.480423 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=25 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:19.907893 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=26 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:20.054061 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25345a7-501e-4de1-8a5e-0fd2cc7edad8 None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:20.055548 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b25345a7-501e-4de1-8a5e-0fd2cc7edad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:20.132063 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25345a7-501e-4de1-8a5e-0fd2cc7edad8 None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:20.137601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b924167-6587-4dd1-a8cb-cfa82b5351ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:20.142641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b924167-6587-4dd1-a8cb-cfa82b5351ee service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:20.142641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b924167-6587-4dd1-a8cb-cfa82b5351ee service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:20.142641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b924167-6587-4dd1-a8cb-cfa82b5351ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:20.142641 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-6b924167-6587-4dd1-a8cb-cfa82b5351ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:20.152049 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b924167-6587-4dd1-a8cb-cfa82b5351ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:20.153720 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b924167-6587-4dd1-a8cb-cfa82b5351ee service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:20.163846 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1440/2894] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:19:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:19:30.158719 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=34 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:33.261372 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294cb48-ada4-4764-b4c8-106f75b010b5 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:33.335761 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294cb48-ada4-4764-b4c8-106f75b010b5 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:33.340702 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9016dab-51fd-4ba1-8401-3b30396de7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:33.341353 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9016dab-51fd-4ba1-8401-3b30396de7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:33.341630 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9016dab-51fd-4ba1-8401-3b30396de7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:33.341902 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9016dab-51fd-4ba1-8401-3b30396de7ed admin admin] PATH_INFO: `/v3/users/a2a1491c324f4306af9f2c96587213cc` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:33.343124 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9016dab-51fd-4ba1-8401-3b30396de7ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2a1491c324f4306af9f2c96587213cc)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:33.345166 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9016dab-51fd-4ba1-8401-3b30396de7ed admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:33.536273 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f9016dab-51fd-4ba1-8401-3b30396de7ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2a1491c324f4306af9f2c96587213cc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:19:33.543510 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f9016dab-51fd-4ba1-8401-3b30396de7ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2a1491c324f4306af9f2c96587213cc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:19:33.624559 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f9016dab-51fd-4ba1-8401-3b30396de7ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2a1491c324f4306af9f2c96587213cc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:19:33.638365 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f9016dab-51fd-4ba1-8401-3b30396de7ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2a1491c324f4306af9f2c96587213cc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:19:33.639406 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f9016dab-51fd-4ba1-8401-3b30396de7ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2a1491c324f4306af9f2c96587213cc'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:19:33.645913 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1455/2895] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:19:33 2022] DELETE /identity/v3/users/a2a1491c324f4306af9f2c96587213cc => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:19:33.650765 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaac490-aa7d-4e84-9fca-90edb94ec84c None None] REQUEST_METHOD: `POST` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:33.651332 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaac490-aa7d-4e84-9fca-90edb94ec84c None None] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:33.651532 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaac490-aa7d-4e84-9fca-90edb94ec84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:33.674156 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.auth.core [None req-fdaac490-aa7d-4e84-9fca-90edb94ec84c None None] MFA Rules not processed for user `31dac45eb8764daf9a148a750fdc78a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92209) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 15:19:33.675397 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-fdaac490-aa7d-4e84-9fca-90edb94ec84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:33.739327 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1441/2896] 10.222.0.94 () {58 vars in 958 bytes} [Tue Jul 26 15:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 15:19:33.749876 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9016dab-51fd-4ba1-8401-3b30396de7ed admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:33.751362 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f9016dab-51fd-4ba1-8401-3b30396de7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:33.814144 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9016dab-51fd-4ba1-8401-3b30396de7ed admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:33.817866 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec6c7c-24eb-42e7-a609-b1c66fe622bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:33.818480 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec6c7c-24eb-42e7-a609-b1c66fe622bf service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:33.818762 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec6c7c-24eb-42e7-a609-b1c66fe622bf service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:33.819062 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec6c7c-24eb-42e7-a609-b1c66fe622bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:33.828664 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec6c7c-24eb-42e7-a609-b1c66fe622bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:33.830090 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec6c7c-24eb-42e7-a609-b1c66fe622bf service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:33.839966 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1456/2897] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:19:34.379866 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaac490-aa7d-4e84-9fca-90edb94ec84c None None] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:19:34.381295 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-fdaac490-aa7d-4e84-9fca-90edb94ec84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:19:34.394347 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaac490-aa7d-4e84-9fca-90edb94ec84c None None] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:19:34.399160 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:19:34.399718 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:19:34.400004 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:19:34.400275 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] PATH_INFO: `/v3/projects/8c2393ea2f0d4b44bc10198887c931b9` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:19:34.401434 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c2393ea2f0d4b44bc10198887c931b9)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:19:34.404205 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:19:34.544449 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c2393ea2f0d4b44bc10198887c931b9', 'request_id': 'req-b4c97130-5d7e-4c8a-abf5-27351e0cda97'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:19:34.550868 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c2393ea2f0d4b44bc10198887c931b9', 'request_id': 'req-b4c97130-5d7e-4c8a-abf5-27351e0cda97'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:19:34.557024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c2393ea2f0d4b44bc10198887c931b9', 'request_id': 'req-b4c97130-5d7e-4c8a-abf5-27351e0cda97'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:19:34.685806 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] The token cache is being invalidate because project 8c2393ea2f0d4b44bc10198887c931b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:19:34.686559 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:19:34.686708 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:19:34.688305 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1442/2898] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:19:34 2022] DELETE /identity/v3/projects/8c2393ea2f0d4b44bc10198887c931b9 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:19:43.835593 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:19:44.681764 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:20:55.147363 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec6c7c-24eb-42e7-a609-b1c66fe622bf service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:20:55.159940 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec6c7c-24eb-42e7-a609-b1c66fe622bf service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:20:55.209514 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9fb9d2-42da-4c56-971a-c2b59bcc433e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:20:55.210122 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fb9d2-42da-4c56-971a-c2b59bcc433e service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:20:55.210400 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fb9d2-42da-4c56-971a-c2b59bcc433e service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:20:55.210672 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fb9d2-42da-4c56-971a-c2b59bcc433e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:20:55.212442 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-aa9fb9d2-42da-4c56-971a-c2b59bcc433e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:20:55.232556 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9fb9d2-42da-4c56-971a-c2b59bcc433e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:20:55.233967 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9fb9d2-42da-4c56-971a-c2b59bcc433e service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:20:55.242056 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1457/2899] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:20:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:21:05.238357 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=32 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:21:11.363480 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:21:11.365517 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:21:11.500880 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c97130-5d7e-4c8a-abf5-27351e0cda97 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:21:11.506664 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34565bba-e6b2-489b-b2ac-a1b1692ac309 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:21:11.507317 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34565bba-e6b2-489b-b2ac-a1b1692ac309 service nova] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:21:11.507638 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34565bba-e6b2-489b-b2ac-a1b1692ac309 service nova] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:21:11.507922 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34565bba-e6b2-489b-b2ac-a1b1692ac309 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:21:11.509743 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-34565bba-e6b2-489b-b2ac-a1b1692ac309 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:21:11.530965 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34565bba-e6b2-489b-b2ac-a1b1692ac309 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:21:11.532605 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34565bba-e6b2-489b-b2ac-a1b1692ac309 service nova] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:21:11.545471 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1443/2900] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:21:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:21:11.922987 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9fb9d2-42da-4c56-971a-c2b59bcc433e service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:21:11.924614 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-aa9fb9d2-42da-4c56-971a-c2b59bcc433e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:21:11.977787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9fb9d2-42da-4c56-971a-c2b59bcc433e service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:21:11.982313 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1b101-7d98-4526-ba82-1248831f02c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:21:11.982931 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1b101-7d98-4526-ba82-1248831f02c1 service neutron] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:21:11.983207 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1b101-7d98-4526-ba82-1248831f02c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:21:11.983533 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1b101-7d98-4526-ba82-1248831f02c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:21:11.985722 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-d0f1b101-7d98-4526-ba82-1248831f02c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:21:12.000565 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1b101-7d98-4526-ba82-1248831f02c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:21:12.003018 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1b101-7d98-4526-ba82-1248831f02c1 service neutron] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:21:12.015598 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1458/2901] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:21:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:21:13.208634 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34565bba-e6b2-489b-b2ac-a1b1692ac309 service nova] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:21:13.210105 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-34565bba-e6b2-489b-b2ac-a1b1692ac309 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:21:13.222368 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34565bba-e6b2-489b-b2ac-a1b1692ac309 service nova] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:21:13.226332 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe83ea-fcdf-4a78-a829-0dadf5d7c3e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:21:13.226928 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe83ea-fcdf-4a78-a829-0dadf5d7c3e9 service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:21:13.227211 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe83ea-fcdf-4a78-a829-0dadf5d7c3e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:21:13.227522 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe83ea-fcdf-4a78-a829-0dadf5d7c3e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:21:13.229811 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-5dfe83ea-fcdf-4a78-a829-0dadf5d7c3e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:21:13.243389 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe83ea-fcdf-4a78-a829-0dadf5d7c3e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:21:13.245474 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe83ea-fcdf-4a78-a829-0dadf5d7c3e9 service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:21:13.257575 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1444/2902] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:21:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:21:14.219499 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1b101-7d98-4526-ba82-1248831f02c1 service neutron] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:21:14.232626 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1b101-7d98-4526-ba82-1248831f02c1 service neutron] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:21:14.238076 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21368631-49cc-46b4-a600-ee05ef45edc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4278fe995e432d88f4346d6f54218b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:21:14.238678 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21368631-49cc-46b4-a600-ee05ef45edc2 service nova] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:21:14.238955 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21368631-49cc-46b4-a600-ee05ef45edc2 service nova] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:21:14.239225 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21368631-49cc-46b4-a600-ee05ef45edc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:21:14.241013 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-21368631-49cc-46b4-a600-ee05ef45edc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:21:14.254085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21368631-49cc-46b4-a600-ee05ef45edc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:21:14.255668 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21368631-49cc-46b4-a600-ee05ef45edc2 service nova] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:21:14.267146 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1459/2903] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:21:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:21:23.253149 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=58 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:21:24.261781 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=28 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:21:28.101647 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe83ea-fcdf-4a78-a829-0dadf5d7c3e9 service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:21:28.111083 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe83ea-fcdf-4a78-a829-0dadf5d7c3e9 service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:21:28.119541 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b569b-2672-4bdf-ac30-8e216230f58b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39fd2708591424ca4892973e7da48ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:21:28.120177 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b569b-2672-4bdf-ac30-8e216230f58b service neutron] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:21:28.120460 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b569b-2672-4bdf-ac30-8e216230f58b service neutron] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:21:28.120735 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b569b-2672-4bdf-ac30-8e216230f58b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:21:28.122690 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-1b5b569b-2672-4bdf-ac30-8e216230f58b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:21:28.226609 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b569b-2672-4bdf-ac30-8e216230f58b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:21:28.228088 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b569b-2672-4bdf-ac30-8e216230f58b service neutron] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:21:28.249225 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1445/2904] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:21:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:21:31.915605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21368631-49cc-46b4-a600-ee05ef45edc2 service nova] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:21:31.917295 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-21368631-49cc-46b4-a600-ee05ef45edc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:21:31.990991 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21368631-49cc-46b4-a600-ee05ef45edc2 service nova] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:21:31.996066 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e204d75-c8e6-4d5d-a89c-83b4c194ebaa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a0728e2f6249bdb45d6f44fb15aa4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:21:31.996683 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e204d75-c8e6-4d5d-a89c-83b4c194ebaa service placement] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:21:31.996961 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e204d75-c8e6-4d5d-a89c-83b4c194ebaa service placement] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:21:31.997235 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e204d75-c8e6-4d5d-a89c-83b4c194ebaa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:21:31.999008 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7e204d75-c8e6-4d5d-a89c-83b4c194ebaa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:21:32.010806 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e204d75-c8e6-4d5d-a89c-83b4c194ebaa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:21:32.012530 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e204d75-c8e6-4d5d-a89c-83b4c194ebaa service placement] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:21:32.022581 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1460/2905] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:21:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 15:21:35.751744 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b569b-2672-4bdf-ac30-8e216230f58b service neutron] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:21:35.763411 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b569b-2672-4bdf-ac30-8e216230f58b service neutron] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:21:35.768740 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159de33-39ab-4b40-85b2-cd4bfb1eb15b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a0728e2f6249bdb45d6f44fb15aa4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:21:35.769469 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159de33-39ab-4b40-85b2-cd4bfb1eb15b service placement] REQUEST_METHOD: `GET` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:21:35.769753 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159de33-39ab-4b40-85b2-cd4bfb1eb15b service placement] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:21:35.770024 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159de33-39ab-4b40-85b2-cd4bfb1eb15b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:21:35.771891 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-3159de33-39ab-4b40-85b2-cd4bfb1eb15b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:21:35.784517 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159de33-39ab-4b40-85b2-cd4bfb1eb15b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:21:35.786096 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159de33-39ab-4b40-85b2-cd4bfb1eb15b service placement] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:21:35.798601 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1446/2906] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:21:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:21:35.880190 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e204d75-c8e6-4d5d-a89c-83b4c194ebaa service placement] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:21:35.881941 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-7e204d75-c8e6-4d5d-a89c-83b4c194ebaa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:21:35.896479 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e204d75-c8e6-4d5d-a89c-83b4c194ebaa service placement] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:21:35.901404 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bff3e3-30a7-4f11-8a43-b4e6e8b83265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:21:35.902071 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bff3e3-30a7-4f11-8a43-b4e6e8b83265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:21:35.902355 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bff3e3-30a7-4f11-8a43-b4e6e8b83265 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:21:35.902631 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bff3e3-30a7-4f11-8a43-b4e6e8b83265 admin admin] PATH_INFO: `/v3/users/c11f43d5614d46cd996789819075520d` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:21:35.903875 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bff3e3-30a7-4f11-8a43-b4e6e8b83265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c11f43d5614d46cd996789819075520d)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:21:35.906294 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bff3e3-30a7-4f11-8a43-b4e6e8b83265 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:21:36.210237 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-60bff3e3-30a7-4f11-8a43-b4e6e8b83265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c11f43d5614d46cd996789819075520d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:36.217318 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-60bff3e3-30a7-4f11-8a43-b4e6e8b83265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c11f43d5614d46cd996789819075520d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:36.337301 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-60bff3e3-30a7-4f11-8a43-b4e6e8b83265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c11f43d5614d46cd996789819075520d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:36.344342 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-60bff3e3-30a7-4f11-8a43-b4e6e8b83265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c11f43d5614d46cd996789819075520d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:36.345031 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-60bff3e3-30a7-4f11-8a43-b4e6e8b83265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c11f43d5614d46cd996789819075520d'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:36.351216 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1461/2907] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:21:35 2022] DELETE /identity/v3/users/c11f43d5614d46cd996789819075520d => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:21:36.627672 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159de33-39ab-4b40-85b2-cd4bfb1eb15b service placement] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:21:36.639427 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159de33-39ab-4b40-85b2-cd4bfb1eb15b service placement] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:21:36.701445 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:21:36.702104 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:21:36.703090 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:21:36.703440 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] PATH_INFO: `/v3/projects/aba3d424c6e64b9fb2b0c5e16eb42763` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:21:36.704723 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba3d424c6e64b9fb2b0c5e16eb42763)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:21:36.706763 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:21:36.987988 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba3d424c6e64b9fb2b0c5e16eb42763', 'request_id': 'req-bb33bf2d-d885-4f62-bb08-84866b09e490'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:36.994937 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba3d424c6e64b9fb2b0c5e16eb42763', 'request_id': 'req-bb33bf2d-d885-4f62-bb08-84866b09e490'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:36.996219 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:21:37.001614 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba3d424c6e64b9fb2b0c5e16eb42763', 'request_id': 'req-bb33bf2d-d885-4f62-bb08-84866b09e490'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:37.119073 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] The token cache is being invalidate because project aba3d424c6e64b9fb2b0c5e16eb42763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:21:37.119405 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:37.120058 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:37.121714 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1447/2908] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:21:36 2022] DELETE /identity/v3/projects/aba3d424c6e64b9fb2b0c5e16eb42763 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:21:37.126559 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bff3e3-30a7-4f11-8a43-b4e6e8b83265 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:21:37.128082 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-60bff3e3-30a7-4f11-8a43-b4e6e8b83265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:21:37.143089 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bff3e3-30a7-4f11-8a43-b4e6e8b83265 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:21:37.200787 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8a0c0-6c29-41be-82d6-dd423f258293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:21:37.201432 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8a0c0-6c29-41be-82d6-dd423f258293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:21:37.201712 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8a0c0-6c29-41be-82d6-dd423f258293 admin admin] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:21:37.201986 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8a0c0-6c29-41be-82d6-dd423f258293 admin admin] PATH_INFO: `/v3/users/bd65b0c1f44c41db8f25fbc9848a9173` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:21:37.203907 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8a0c0-6c29-41be-82d6-dd423f258293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd65b0c1f44c41db8f25fbc9848a9173)` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:21:37.205843 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8a0c0-6c29-41be-82d6-dd423f258293 admin admin] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:21:37.208369 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:21:37.416005 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f1b8a0c0-6c29-41be-82d6-dd423f258293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd65b0c1f44c41db8f25fbc9848a9173'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:37.425258 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f1b8a0c0-6c29-41be-82d6-dd423f258293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd65b0c1f44c41db8f25fbc9848a9173'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:37.558605 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f1b8a0c0-6c29-41be-82d6-dd423f258293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd65b0c1f44c41db8f25fbc9848a9173'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:37.579713 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f1b8a0c0-6c29-41be-82d6-dd423f258293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd65b0c1f44c41db8f25fbc9848a9173'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:37.580537 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.notifications [None req-f1b8a0c0-6c29-41be-82d6-dd423f258293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd65b0c1f44c41db8f25fbc9848a9173'} {{(pid=92208) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:37.586665 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1462/2909] 10.222.0.94 () {58 vars in 1224 bytes} [Tue Jul 26 15:21:37 2022] DELETE /identity/v3/users/bd65b0c1f44c41db8f25fbc9848a9173 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:21:37.928561 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] Authenticating user token {{(pid=92209) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:21:37.930031 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.fernet_utils [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92209) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:21:37.943791 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33bf2d-d885-4f62-bb08-84866b09e490 admin admin] Validating token access rules against request {{(pid=92209) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:21:38.003066 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56380e6-480d-45cf-bc2d-d5616fd0fec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31dac45eb8764daf9a148a750fdc78a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '382829af20d649b9beb375e3c3cc2d50', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92209) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:21:38.003807 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56380e6-480d-45cf-bc2d-d5616fd0fec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:21:38.004019 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56380e6-480d-45cf-bc2d-d5616fd0fec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:21:38.004296 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56380e6-480d-45cf-bc2d-d5616fd0fec1 admin admin] PATH_INFO: `/v3/projects/1e0ee14bac364a66b46a76f74172c6ad` {{(pid=92209) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:21:38.005466 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56380e6-480d-45cf-bc2d-d5616fd0fec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e0ee14bac364a66b46a76f74172c6ad)` {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:21:38.007424 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56380e6-480d-45cf-bc2d-d5616fd0fec1 admin admin] RBAC: Authorization granted {{(pid=92209) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:21:38.163657 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e56380e6-480d-45cf-bc2d-d5616fd0fec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e0ee14bac364a66b46a76f74172c6ad', 'request_id': 'req-e56380e6-480d-45cf-bc2d-d5616fd0fec1'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:38.170093 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e56380e6-480d-45cf-bc2d-d5616fd0fec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e0ee14bac364a66b46a76f74172c6ad', 'request_id': 'req-e56380e6-480d-45cf-bc2d-d5616fd0fec1'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:38.177135 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e56380e6-480d-45cf-bc2d-d5616fd0fec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e0ee14bac364a66b46a76f74172c6ad', 'request_id': 'req-e56380e6-480d-45cf-bc2d-d5616fd0fec1'} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:38.304747 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e56380e6-480d-45cf-bc2d-d5616fd0fec1 admin admin] The token cache is being invalidate because project 1e0ee14bac364a66b46a76f74172c6ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92209) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 15:21:38.305016 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e56380e6-480d-45cf-bc2d-d5616fd0fec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:38.305660 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG keystone.notifications [None req-e56380e6-480d-45cf-bc2d-d5616fd0fec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92209) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 15:21:38.307264 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: [pid: 92209|app: 0|req: 1448/2910] 10.222.0.94 () {58 vars in 1233 bytes} [Tue Jul 26 15:21:37 2022] DELETE /identity/v3/projects/1e0ee14bac364a66b46a76f74172c6ad => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 15:21:47.583072 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:21:48.301148 dv-cinder-850668-2-iscsi devstack@keystone.service[92209]: DEBUG dbcounter [-] [92209] Writing DB stats keystone:DELETE=10,keystone:SELECT=55,keystone:INSERT=2 {{(pid=92209) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 15:22:59.710264 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8a0c0-6c29-41be-82d6-dd423f258293 admin admin] Authenticating user token {{(pid=92208) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 15:22:59.711751 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-f1b8a0c0-6c29-41be-82d6-dd423f258293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:22:59.806616 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8a0c0-6c29-41be-82d6-dd423f258293 admin admin] Validating token access rules against request {{(pid=92208) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 15:22:59.866085 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1d365-11b4-424e-960e-e425dd4cf26c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42a0728e2f6249bdb45d6f44fb15aa4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9b328041da24cedbc801b271d6e0419', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92208) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 15:22:59.866737 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1d365-11b4-424e-960e-e425dd4cf26c service placement] REQUEST_METHOD: `GET` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 15:22:59.867020 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1d365-11b4-424e-960e-e425dd4cf26c service placement] SCRIPT_NAME: `/identity` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 15:22:59.867311 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1d365-11b4-424e-960e-e425dd4cf26c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92208) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 15:22:59.869196 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.fernet_utils [None req-6cf1d365-11b4-424e-960e-e425dd4cf26c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92208) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 15:22:59.880811 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1d365-11b4-424e-960e-e425dd4cf26c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 15:22:59.882383 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1d365-11b4-424e-960e-e425dd4cf26c service placement] RBAC: Authorization granted {{(pid=92208) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 15:22:59.893002 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: [pid: 92208|app: 0|req: 1463/2911] 10.222.0.94 () {60 vars in 1365 bytes} [Tue Jul 26 15:22:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 15:23:09.888151 dv-cinder-850668-2-iscsi devstack@keystone.service[92208]: DEBUG dbcounter [-] [92208] Writing DB stats keystone:SELECT=67 {{(pid=92208) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}